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NOTE TD READER 
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Analysis, by John E. Tolle. 

Volume II. Online Catalog Use: Results of Surveys and Focus Group 
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Volume III. A Comprehensive Study of Online Public Access Catalogs: 

.An Overview and Application of Findings, by Neal K. 
Kaske and Nancy P. Sanders. 

In addition to these volumes, the reader may wish to consult the final [sports 

prepared by the organizations which were also funded by the Council on Library 

Resources, Inc. in this online public access catalog study. Following is a 
list of these reports: 

Research Libraries Group, Inc. Public online catalogs and research 
libraries: final report to the Council on Library Resources. Stanford, 
CA: Research Libraries Group, Inc., 1982 September. 

Matthews, Joseph K. A study of six online public access catalogs: a 
final report submitted to the Council on Library Resources, Inc. brass 
Valley, CA: J. Matthews & Associates, Inc.; 1982 November. 

University of California, Division of Library Automation and Library 
Research and Analysis Group. Users look at online catalogs: results of 
a national survey of users and non-users of online public access 
catalogs: final report to the Council on Library Resources. Berkeley, 
CA: University of California; 1982 November 16. 

Anderson, Rosemary; Reich, Victoria A.; Wagner, Pamela Roper; Zich, 
Robert. Library of Congress online public catalog users survey: a 
report to the Council on Library Resources. Washington, DC: Library of 
Congress, Office of Planning and Development; 1982 October 29. 
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ABSTRACT 



This report covers the background, scope, and findings of a mul tifaceted 
research project proposed to the Council on Library Resources, Inc. (CLR), and 
entitled "Online Public Access Systems: Data Collection and Analysis", The 
overall research effort was sponsored by CLR, undertaken by OCLC, and 
described in a three-volume final report. The ultimate goal of this project 
is to improve the library patrons' ability to access information through the 
design and enhancement of online public access catalogs (OPACs). The first 
part of this project describes the current use and patterns of use of OPACs 
through transaction log analysis (see Vol, I), The second part describes 
library patrons' needs and perceptions-of OPACs thraugh analysis of 
questionnaire data and focus group interviews (see Vol, II), The data was 
collected nationally in cooperation with four other organizations under 
similar CLR graqts or contracts. Vol. Ill evaluates and integrates the 
findings about; online public access catalogs to assist library decision makers 
when .planning for the initial installation of OPACs or the enhancement of 
existing OPACs. \ 
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1.0 INTRODUCTION 



1.1 GOAL 

The ultimate goal of this research is to improve the 1 ibrary patron 1 s ability 
to access information through the design and enhancement of onl i t ne. catalogs. 
This multi faceted research effort has provided new insights into the library . 
patron's perceptions about online public access catalogs (OPACs) and has 
improved our understanding of how OPACs are used, thus moving us closer to our 
goal. Just over a year ago at a symposium o>i online catalogs prior to the 
American Library Association's midwinter meeting, it was estimated that, only 
1% of the needed information on the subject of OPACs was available. Today, as 
a result of this and related studies, we are steadily i ncreasing our knowledge 
about this special human-computer interaction thanks to the foresight and 
sponsorship of the Council on Library Resources (CLR) . 

Our goal, to improve the library patron's ability to access information 
through the design and enhancement of online catalogs, is shared by a number 
of library organizations. The key organization, is CLR, which sponsored the 
OCLC research, as well as three other grants and one contract, all dealing 
with OPACs, These related grants and contract were awarded to (1) The Library 
of Congress, Office of Planning and Research; (2) University of California, 
Office of the Assistant Vice-President for Library Plans and .Policies; (3) 
Research Libraries Group, Inc.; and (4) J. Matthews and Associates, Inc. 

OCLC, in cooperation with the four other organizations developed survey 
instruments, i.e., self-administered questionnaires for users and nonusers of 
OPACs, to learn about patrons' perceptions of Of>ACs at libraries across the 
nation. In addition, the OCLC research team employed two other methodologies 
to supplement what could^be learned through survey research and to provide a 
greater understanding of the use made of online catalogs. These two 
methodologies are (1) focus group interviews; and (2) transaction log 
analysis. 

Together, the three methodologies provide a balanced perspective of online 
patron behavior. The self-administered questionnaires provide a large amount 
of data within a short period of time at low cost. This data must be ■" 
validated from other sources, however, because the questionnaire responses 
provide little more than controlled feedback. The focus group interviews were 
employed to validate and to further expand on the information gathered from 
the questionnaires. The discussions, guided as little as possible by the 
researcher/moderator once the topic has been introduced, were used to test' 
hypotheses posed by the research team concerning the use or nonuse of OPACs by 
library patrons and staff. The transaction log is a machine record of the 
interaction between computer and patron. By analyzing transaction logs we can 
visualize patron's input access points and discover patterns of use. 
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Figure 1 pictures the balancing nature of the three methodologies. When 
mapped on the feedback accuracy/cost efficiency continuums, the survey 
questionnaire falls at the low end of -the accuracy scale, but the high end of 
the cost efficiency scale. Transaction log analysis falls at the opposite end 
of each scale, while the focus group interviews fall in the middle. 

The three methodologies (survey research, focus group interviews, and 
transaction log analysis) also make it possible to address the first two of 
four key priorities set during the "Dartmouth Conference," a CLR-sponsored 
research project in 1980 which brought together the chief information 
professionals involved in the design, planning, implementation, and 
enhancement of online public access catalogs. The four priorities were: (1) 
analyzing user requirements and behavior; (2) monitoring existing public 
access systems; (31 developing methods for cost management; and, (4) 
developing distributed computing and system links. [4] 



High 



Low 



Cost Efficiency for Data Collection 
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Focus group 



Transaction 



Questionnai re 



Interviews 



Log Analysis 



Feedback Accuracy 



Low 



High 



Figure 1. Feedback Accuracy Continuum 
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1.2 A CONCEPTUAL MODEL FOR ONLINE CATALOG RESEARCH 

Several models have been constructed to aid our understanding of the 
human-computer interactions taking place when a library patron uses an OPAC. 
These models have helped identify the variables that need to be studied to 
bring about an understanding of the user's requirements, behaviors, and ■ 
expectations for the OPACs. Figure 2 portrays a model with four major 
components: the library; the librarian on duty; the computer system with its 
database(s), displays, and dialog;. and the patron. The elements listed under 
each of these four key components are variables we could and in some cases 
have selected to study. Central to the OCLC study is the interaction between 
the system and the patron. A model developed by Charles Hildreth (see Figure 3) 
clearly illustrates the user interface for an interactive system. 
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ONLINE 
CATALOG 



THE SYSTEM 



Data Base(s) 




Displays 




Command Language(s) 






Files 


Formats 


Number of conmands 




Access Methods 


Codes 


Complexity 




Thesaurus 


User aids 


Consistency 




Records 


Error messages 


Flexibility 




Etc. 


Hard and/or soft copy 


Menus and prompting 






Etc. 



THE PATRON 





Knowledge of the system 




Skill in using the system 




Attitudes toward 




The library's collection 




The system 




The library staff 



Note: This chart is based in part on Fenichel, Library Research 2:107-127 (1980-81), 
and notes of Pauline (Atherton) Cochrane. 
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Figure 2. Online Public Access Catalog: Factors 
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Experience 
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Information needs 
Model of system 
structure 
and operations 
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The DIALOGUE locu< 



DESIGNERS 



(Rules/conventions for description, 
subject analysis, classification, 
access points, syndetics, 
authority control, 
output format, etc.) 



Telecommunications, applications, 
and database management . . . 

HARDWARE AND SOFTWARE 



t 



HUMAN CATALOGER/INDEXER 



THE DATABASE 

(Indexes, authority, 
and bibliographic files) 

SYSTEM 
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(Source: Hildreth, Charles R. Online public access catalogs: 
the user interface. Dublin, OH: OCLC; 1982. Figure 5, p. 42.) 



Figure 3. The User Interface in' Interactive Systems 
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1.2.1 OCLC's Research Plan 

This research project is divided into three parts. In the first, the use made 
of OPACs by library patrons was studied by analyzing transaction logs 
(Vol. I). These logs are the machine-readable records of the activity that 
takes place between patrons and the OPAC, They catalog the operations patrons 
actually performed at the terminal, not what they believed they did. 

The second part of the research project studied the needs and perceptions of 
library patrons (see Vol. II). Two data collection methodologies were used: 
(1) self-administered questionnaires; and (2) focus group interviews. The 
third part of the project evaluated and integrated the findings from the first 
two parts and presented them in format and language for the nontechnician, 
particularly the library manager. 

As a project summary, Table 1, taken from the original OCLC proposal to CLR, 
is presented. below. It lists the parts of the project, the research 
questions, related studies, the research tasks, the methods to be used, and 
the project output. The objectives and tasks for each of the three parts 
follow in an outline form. 
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Table 1. OCLC Proposed Study of Online Public Access Catalogs 



Parts 

1. Study and - 
analyze current 
systems* design 
using 

transaction 
logs. 



2. Study and 
analyze patterns 
of use (from 4 
systems* daily — 
logs). 



3. Study and 
analyze patron 
perceptions. 



4. Experiment: 
Develop an 
Interface 
simulator; test 
new and old 
features. 



5. Evaluate and 
Integrate 
findings for 
management 
decisions. 



Research Questions 
and Related Studies 

Are sophisticated 
design features 
actually used? 
(Continue work on 
normalized commands 
and definitions as 
established in the 
current CLR-fUnded 
study. ) 



Tasks 

Obtain, analyze, 
categorize, compare, 
and report the 
current 

function/command, 
utilization of 
several online public 
catalogs. 



Are similar patterns Identify and report 
found across systems? the individual 
Between online library patron *s 

cfttftlng* and o nline current use pat terns 



reference/search 
services? (Compare 
with NLM/OCLC study.) 

How do patrons react 
to online catalogs? 
What additional or 
different features do 
patrons require? 



What effect does 
varying interface 
features have on 
patron use of online 
catalogs? 



What are the most 
useful features 1n an 
OPAC? How can 
improved OPACs be 
developed, tested, 
and Implemented 
cost-effectively? 
What are the 
requirements for an 
OPAC— staff, online 
training and aids, 
equipment, and space? 



of online public 
catalogs. 



Assess, analyze, and 
report the current 
perceptions of 
library users and 
nonusers of existing 
OPACs. 

Create and use 
experimental 
procedures to test 
various versions of 
online library 
catalogs, without 
redesign or 
interruption of 
existing online 
systems. 

Integrate and apply 
the findings of the, 
above studies to 
develop improved 
online catalogs. 
Disseminate findings 
and recommendations 
to library managers 
and other OPAC 
designers. 



Methods Project Output's 

At system level, Comparative data 

obtain gross across systems 

statistical measures 

to establish relative 

use of commands 

within each of 10 

systems. 

1. Correlation 
analysis 

2. Frequency analysis 

Trace sequence of , Data on patron 
actions in terminal use or different 
sessions to establish, .systems 
Ind ividual user 



patterns within 4 
systems. 



Focused-group 
Interviews at'6 
libraries and data 
from RLG/CLR study. 



Use Interface 
simulator to model 
and monitor OPAC 
features. 



Data on patron 
perceptions of 
OPACs 



Data on 

effectiveness of 
OPAC features 



Publish results, hold 
target-audience 
workshops, provide 
feedback to OCLC, 
ARL, CLR, etc. 



New data, and 
research tool (s) 
for the 
information 
science field 
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" "" Objectives" andTasks for Part ti ~ 

Transaction Log Analysis 

Objective: To discover the extent to which current systems' features are used 
and patterns of use. 

Task 1A Acquire activity reports from current online public catalog systems 
Task IB Categorize and analyze the activity reports 
Task 1C Normalize the activity reports 

Task ID Design a format for presenting the different activity statistics 

Task IE Evaluate transaction activity logs 

Task IF Obtain transaction tapes from four systems 

Task 1G Extract a sample week's activity 

Task 1H Analyze typical use patterns 

Task II Analyze transaction information 

Research efforts are always limited by time and money. This project was no 
exci tant Therefore, Tasks 1A through Task ID were dropped from the project 
except where some of the information could be presented as a by-product of ^ 
Tasks IE through Task II. '\ 

Objectives and Tasks for Part II. 
Current Perceptions of New Forms of the Library Catalog 

Objective: To evaluate and eventually improve online public catalogs from an 
analysis of patron needs and perceptions of online catalogs. 

Task 2A Train interviewers and distribute user perception questionnaires 

Task 2B Coordinate the administration of the questionnaires 

Task 2C Analyze the questionnaires 

Task 2D Conduct focus group interviews 

All of these tasks were conducted as proposed. 



20 



Report Number: 0CLC/0PR/RR-83/4 
1983 March 31 



Objectives and Tasks for Part III. 
Application of Findings 



Objective- There were two specific objectives stated for this segment of the 
project. They are: (1) application of the project findings to the design and 
improvement of the online public access catalog by disseminating the research 
results; and (2) application of the research findings to library problems 
related to introducing the online public access catalog. 

Task 3A Disseminate findings: the change process and project reports 

Task 3B Apply and evaluate results: advisory panel 

Task 3C Project management: organization, schedule, and consultants 

The scope of Part 3 of the project was narrowed by omitting Task 3B and 
cutting back in both Tasks 3A and 3C. One of the items omitted from this 
segment of the project, an invitational conference on various aspects of the 
-online public access catalog, was later conducted by CLR with an expanded 
format^ Part 3 of this project also covered, the administrative components of 
the project as a whole. 

The research design and uses of the data and information derived from this 
project (proposed and as completed) included the following: 

1. Terminal use data from existing OPACs to show in general terms the use of 
OPAC systems; 

2. Transaction data analysis from existing OPACs to/ provide information on 
individual patron use of the systems; 

3. Patron interview and questionnaire data to^rovide user reactions to the 
systems; 

4. Synthesis of findings and a translation of results into recommendations 
for improvements in the design of online public access catalogs; 

5. Solutions for related library problems, such as number of terminals, 
sophistication of command system, user education, staffing, support 
equipment, and space required; 

6. Dissemination of results to library managers, systems designers, and 
interested professionals through published literature, workshops, and 
conferences. 
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1.2.2 Integration with Other CLR Projects 

In addition to the OCLC research, the Council on Library Resources 
concurrently funded four other studies on the topic of OPACs. Each of the 
five organizations used one research tool, the survey questionnaire, in 
common. (Thus, the same questionnaire was employed at 29 institutions.) The 
OCLC research team supplemented this method by using other methodologies, 
viz., focus group interviews and transaction log analysis. The reports of 
other organizations funded by CLR covering questionnaire results are: 



1. J. Matthews and Associates, Inc. A Study of Six Online Public Access 
Catalogs : A Review of Findings . November 19"8~27 IT] V 

2. Library of Congress. Library of Congress Online Public Access, Catalog 
Users Survey : A Report to the Council on Library Resources . October 
■T98"27 ITT u 

3. Research Libraries Group, Inc. Public Online Catalogs and Research 
Libraries . September 1982. [5] 

4. University of California, Division of Library Automation and Library % s ^ 
Research and Analysis Group. Users Look at OnlineXata}bgs : Results of 
National Survey of Users and Nonusers~ "oT TiFline Public Access Catalogs . 
November 1982. L6J * A 



These four reports and the three volumes of OCLC's final^ report will be made 
available through the ERIC Clearinghouse approximately six months after the 
date' of publ i cation. /"^X-i 



The grant to OCLC for this project was made in two parts. The first part was 
an officer's grant whose results are reported in Pilot Test of the Online 
Publ i c Access Catalog Project's User and Nonuser Questionnai res .L2J 

Table 2 explains the integration of the CLR projects with OCLC's related 
research projects. 
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Table 2. Related OCLC Research Projects 



PROJECTS 

Online Patron Access to 
Bibliographic Data Bases. 
Council on Library 
Resources, Inc. 
1980 Jine - Aug. 
< • / f Ni* 



FOCUS/PURPOSE 

Studied direct access by 
library users to online 
bibliographic data bases 



CONTRIBUTION 

TO THIS PROPOSAL 

Needs and priorities 
identified for 
further research 



P.I.// 



Kaske 



*N0TE: THIS WAS THE FIRST GROUP PROJECT FUNDED BY CLR WHICH WAS A JOINT EFFORT OF 
OCLy AND R.L.G. D. FERGUSON WAS THE PRINCIPAL INVESTIGATOR FOR R.L.G. THE 
REPORT OF THIS PROJECT IS KNOWN AS THE "DARTMOUTH CONFERENCE" REPORT. 



Bibliographic Electronic Home Developed and tested 

Information Service: Channel home information 

2000. service 

OCLC, Inc. 

1980 March to 1981 

P.I./T. Harnish . 



Menu-driven system; 
patron access; user 
aids for online 
searching 



Modeling and Evaluation of 
Online User-Behavior. 
National Library of Medicine 
1979 Dec. - 1981 Sept. 
P.I*/ W. D. Penniman 



Examined and modeled 
user behavior with 
online search service 



Pattern analysis of 
user transactions on 
an information 
retrieval system 



Terminal Requirements for 
Online Catalogs in Libraries. 
National Science Foundation 
1979 May - 1981 Nov. 
P.I./ N. Kaske 



Measured and analyzed 
four aspects of library 
use to predict online 
catalog use 



Some understanding of 
online catalog use 



Online Public Access Systems: 
Data Collection Instruments 
for Patron and System 
Evaluation. 

Council on Library Resources 
1981 Jan. - June 
P.I./ N. Kaske; C. Hildreth, 
Project Manager 



Summarized and analyzed 
functions, command 
capabi li ties, semantics, 
and syntax 



Standard definitions 
and categories of 
commands'; patron 
assessment tool 



*N0TE: THIS WAS THE SECOND GROUP PROJECT FUNDED BY CLR WHICH WAS -A JOINT EFFORT 
OF OCLC, R.L.G. , U.C.D.L.A., AND J.R. MATTHEWS & ASSOC. 



Subject Access. 
•1983 Feb. 
OCLC, Inc. 
P.I./N. Kaske 



Examined patrons' 
perceptions, 
expectations, and 
criteria for success in 
using subject catalog; 
mental processes used in 
subject searches 



Assessment of patron 
problems with current 
methods of subject 
access and patron- 
suggested solutions; 
experience in using 
focused-group and 
open-ended interviews 
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It is the objective of this chapter to provide a nontechnical overview of \he 
CLR-funded OCLC study of OPACs and thereby to answer a number of basic \ 
questions about the research undertaken. Some of these questions are: Wha£ 
data was collected?; Why was this data collected?; How was the data analyzed?; 
What were the findings? 

Three research tools were employed by the OCLC research team to collect the 
needed data on OPAC use and patterns of use: questionnaires, focus group 
interviews, and transaction log analysis. The first, a survey questionnaire, 
had the advantage of providing a large amount of data in a short period of ^ 
time at low cost. The major disadvantage is that it only provides for \ ^ 
controlled feedback. That is, respondents can answer only the questions asked 
and can/answer those questions only within the limits of the options provided 
by the ^questionnaire. * > ' 

The focus group interview, on the other hand, does not force options from the 
interviewers, but permits them to explore ideas in an unstructuredlgroup 
discussion. The researcher uses open-ended questions to initiate the 
discussion and then can follow up on unclear areas. Its major disadvantage is 
high cost and the level of interview skills required of the researcher. 

Transaction log analysis is the machine-readable record of the patron's search 
and/or a record of the system's responses. It is a record, therefore, of what 
happened, not what someone remembers. The chief advantage of transaction log 
analysis is that it covers all basic facts of the interaction between the 
patron and the system. Its primary disadvantage is that few systems are 
currently capable of recording the interactions between the patron and the 
system. The monitoring creates an overhead cost in system-time and usually 
requires additional equipment. Furthermore, it is unlikely that most OPAC 
systems will provide this capability in the future as a monitoring system adds 
significantly to the cost of an OPAC and is advantageous, for the most part, 
only in systems research such as this. That is, the information provided is 
not particularly useful to those using or operating the system on a daily 
basis. When used together, the three methods provide a clear picture of OPAC 
use, as the disadvantages of one research tool tend to be balanced by the 
strengths of the others. 
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As implemented, the research design was somewhat more complex, though likely 
stronger, due to the cooperative nature of the questionnaire-related portion 
of the study. As noted previously, the Council on Library Resources funded 
four organizations, in addition to OCLC, to conduct a thorough national survey 
of OPAC use via questionnaire. The four other organizations are: Library of 
Congress, Office of Planning and Development; Research Libraries Group, Inc. 
(RL6); University of California, Division of Library Automation; and J. 
Matthews and Associates, Inc. The survey included both users and' nonusers of 
OPACs. The questionnaires were developed by a committee composed of the 
principals of the five cooperating organizations. The development process was 
difficult at times but a useful product was generated. The printing of 
questionnaires was coordinated by Douglas Ferguson of RLG and Gary Lawrence of 
UC/DLA. The questionnaires were- designed to be machine-readable and were 
processed at Stanford University. The basic statistical analysis was done at 
UC/DLA. 

There were 13,591 users (i.e., patrons who had been observed using an online 
catalog) who were approached and asked to complete a questionnaire.' Of these 
13,591 some 8,094 agreed to participate and completed a user questionnai re. 
Some 7,625 library patrons (i.e., patrons who were observed entering the 
library) responded negatively when asked if they had ever used the OPAC; 3,981 
agreed to participate and completed the nonuser questionnaire. A copy of the 
final user questionnaire distributed in the study is provided in Appendix A; a 
•copy of the nonuser questionnaire in Appendix B. Copies of the user and 
nonu.ser questionnaires administered in the pretest are displayed in Appendixes 
C and D, respectively. Slightly more than 50 percent of the users and 35 
percent of the nonusers who were approached completed the OCLC-adrtii nistered 
questionnaire. 

i — 

Acceptance rate and productivity data for" user and nonuser questionnaires from 
the 9 participating libraries are reported in Tables 3 and 4. Thenumber of 
patrons approached at eacl^ library and the number completing questionnaires 
are reported. Table 5 repprts the sample size by the type of library sampled. 

The organization of the OCLt efforts was divided into three parts. The 
transaction log analysis section was managed tay>Dr. John E. Tolle and is 
reported in Vol. I of this flhal report. The focus group interview and 
questionnaire sections were managed by Dr. Karen Markey and are reported in 
Vol. II. The administration and application of the findings were the 
responsibility of Dr. Neal K. Kaske, Principal Investigator of the project. 
Both Dr. Tolle and Dr. Markey were Project Managers for their respective parts 
of the study. 
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Table 3. User Questionnaire Response Rate and Productivity Data 



Library or 
Library System 


No. of 

Time 

Blocks 


No. of 
No. of Completed 
Scheduled Questionnaires 
Hours Per Hour 


Complete 


Incomplete 


Total 

Acceptance 


Decline 


Total 
Valid 


Acceptance 
Rate {%) 


Completion 
Rate (%) 


Response 
Rate (X) 


No. 

Successful ly 
Scanned 


Syracuse University 
Apr. 1- Apr. 21, 1982 


84 


252 


5.1 


12B6 


83 


1369 


647 


2016 


66 


94 


64 


1339 


Ohio State University 
Apr. 6- May 28, 1982 


131 


393 


3.5 


1369 


111 


1480 


1774 


3254 


45 


93 


42 


1247 


Dallas Public Library 
Apr. 1- Nay 29, 1982 


198 


488 


2.1 


1019 


100 


1119 


349 


1468 


76 


91 


69 


1081 


Iowa City Public Library 
Apr. 5- Nay 29, 1982 


32 


67 


5.8 


3B6 


33 


419 


136 


555 


76 


92 


70 


410 


University of Akron 
Apr. 1- May 14, 1982 


54 


505 


.25 


127 


7 


134 


35 


169 


79 


95 


75 


132 


Ohio University 

Aor. 12- Mav 23 1982 

■ * Aft* m J It J| i JUL 


27 


54 


.89 


48 


4 


52 


20 


72 


72 


92 


67 


51 


State Library of Ohio 
Apr. 29- June 2, 1982 


7 


41 


.12 


5 


0 


5 . 


3 


8 


63 


100 


63 


5 


University of Texas 
at Austin 

Apr. 5- May 14, 1982 


35 


261 


.39 


102 


3 


105 


30 


135 


78 


97 


76 


104 


University of Texas 
at Dallas 

Apr. 10- May 2B, 1982 


14 


139 


.27 


36 


1 


37 


9 


46 


80 


97 


78 


37 






Total 


User Questionnaires 

Accepted ■ 




4720 






Total User Questionnaires 
Scanned Successfully 




4406 
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Table 4. Nonuser Questionnaire Response Rate and Productivity Data 



Library or 
Library System 


No. of 
Time 

Dl OCK S 


No. of 

Scheduled 

Hours 


No. of 

Completed 

Questionnaires 

Per Hour Complete 


Incompl ete 


Total 

Acceptance 


Decl ine 


Total 
Val id 


Acceptance 
Rate (%) 


Completion 
Rate (%) 


Response 
Rate (%) 


No. 

Successfully 
Scanned 


Syracuse University 
Apr. 2- Apr. 17, 1982 


18 . 




5.3 


288 


c 
o 


294 


160 


454 


65 


98 


63 


294 


Ohio State University 
Apr. 13- Hay co» 1982 


16 


48 


5.7 


275 


7 


282 


85 


367 


77 


98 


75 


282 


Dallas Public Library 
Apr. 3- June 2, 1982 


33 


65 


4.5 


292 


12 


304 


97 


401 


76 


96 


73 


299 


Iowa City Public Library 
Apr. 19- May 24. 1982 


11 


23 


6.2 


142 


3 


145 


55 


200 


73 


98 


71 


144 


University of Akron 
Apr. 2- Apr. 21, 1982 


4 


12 


8.3 


100 


0 


100 


24 


124 


81 


100 


81 


100 


Ohio University 

Apr. 10- Apr. 26, 1982 


7 


14 


7.4 


104 


2 


106 


52 


158 


67 


98 


66 


106 


State Library of Ohio 
Apr. 10- June 2. 1982 


21 


80 


i.r 


86 


0 


86 


37 


123 


70 


100 


70 


66 


University of Texas 
at Austin 

Apr. 5- Apr. 16, 1982 


7 


14 


7.3 


103 


2 


105 


32 


137 


77 


98 


75 


103 


University of Texas 
at Dallas 

Apr. 14- Apr. 30, 1982 


7 


13 


8.0 


103 


1 


104 


20 


124 


84 


99 


83 


104 






Total 


Nonuser Questionnaires 
Accepted - 




1526 






Total Nonuser Questionnaires 
Scanned Successfully ■ 


1518 
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Table 5. Number of Questionnaires Completed by Users and Nonusers 

Categorized by Type of Library 



Type of Library 



Users 



Number 



NonUsers 
Number % 



ARL Libraries 
Other Academic Libraries 
Community College Libraries 
State and Federal Libraries 
Public Libraries 



4,701 
661 
155 
530 

2,047 



58 
8 
2 
7 

25 



1,529 
764 
228 
384 

1,076 



38 
19 
6 

10 
27 



TOTAL 



8,094 



100 



3, 981 



100 



2.1 OBJECTIVES AND METHODOLOGIES USED IN THE PROJECT 

The objective for Part 1 of this study (see Vol. I) was to discover the extent 
to which current systems' features are used and the patterns of their use. To 
focus our study in this new field of research (i.e., human-computer 
interaction/online public access catalogs) we posed several research questions 
for investigation. These questions were: 

1. What is the frequency of "0 postings" searches in which searchers re-enter 
or revise their query? 

2. Are there sophisticated searching features available to patrons (such as 
logical operators, word adjacency, multiple-field term searching) which 
are used infrequently? What correlation, if any, does type of library 
have with needed prompting devices or search sophistication? 

3. What is the relative frequency of available fields searched? 

4. What is the relationship between rigid command syntax and command usage? 

5. What is the relationship between rigid command syntax and error message 
frequency? 

6. What, if any, patterns of feature use exist, e.g., are author searches 
usually followed by title searches? 

7. What is the comparison of user time at the terminal for subject searches 
to author and/or title searches? 

8. What kind of relationship exists between errors and length of searches? 
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To answer these questions we proposed to collect data for two months at each 
of four different locations. In reality we were able to collect data from 
four locations, but were unable to obtain two months of data for each location 
because of the volume of transactions or the impact of transaction log 
analysis on individual OPACs. The locations and period of data collection are 
listed beslow. 

Library of Congress - March to June 1982 

Syracuse University - April 12 to April 28, 1982 

Dallas Public Library - August 9 to August 21, 1982 
The Ohio State University - January to December 1981, March to May 1982, 

and November 1982 

In each of the libraries extra work was required to supply this data; in some 
cases a great deal of extra effort. Because of the level of effort and the 
compressed timing of the project, not all the data was supplied as requested. 
Participating libraries were very cooperative, but the creation of this data 
took much time and money. Ideally, available data would permit the creation 
of a monitor record. The data elements in this monitor record are reported in 
Figure 4. The first element, "session identifier," which divides the activity 
of one patron from another, was not directly available for any of the systems 
although most institutions helped to create this "session identifier." Many of 
the other elements were also not available but the data provided has proved to 
be valuable and informative. Table 6 reports each of the data elements that 
were available on a system by system basis. 

The objective of Part 2 of this research project (see Vol. Il) was to evaluate 
and eventually improve online public catalogs from an analysis of patron needs 
and perceptions of online public access catalogs. Some of the research 
questions answered in the investigation were: 

1. How do library patrons react to online catalogs? 

2. When do OPAC users prefer to use the card catalog? 

3. In what ways are library patrons satisfied with their online public 
catalog searches? 

4. What improvements are necessary to attract nonusers to search online 
catalogs? 

5. What additional or different features do patrons require of online 
catalogs? 

These and many other questions have been addressed by using both the 
questionnaires and the focus group interviews. The questionnaires were first 
drafted in early 1981, then pilot-tested during the fall, and distributed 
again in spring 1982 after revisions to the pilot-test instruments. 
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1. Session identifier . The unique identifier associated with t*he 
particular session being monitored. 

2. User identifier . The unique identifier associated with the 
individual user being monitored. 

3. Database or file being used . If relevant, the name of the file 
currently being accessed by the user in a given database. 

4. Date . 

5. Time stamp . The time at which each transaction occurs. Time 
stamps should provide as much accuracy as possible, although a 
time stamp resolution exceeding hundredths of a second is not 
generally useful. The point at which the time stamp will be 
applied must be specified. Ideally, the input time stamp should 
be applied when the user completes the input (e,g., depresses the 
ENTER, RETURN, or other special function key), and the output time 
stamp should be applied when the first character of output is - 
delivered to the user. Since these exact times are not often 
available how the time stamps differ from the ideal time stamps 
should be stated. 

6 The sou rce of each transaction . Possible sources should include 
at least the terminal user, system, and other transaction source 
(e.g., stored command files or operator messages). 

7 Svstem-depende nt state information . If other information about 

the transaction is read ily available, it should be included. Common 
examples include a transaction code generated to govern internal 
processing or special error or return codes. 

8. Blank space . This is needed for state code assignment during post- 
session analysis. 

9. Length of text portion . Number of characters in the input or 
response. 

10. Te xt portion . Contains the text of the user input or the system 
response, the complete text is preferred when practical, but it 
may be truncated. 



Figure 4. Recommended Monitor Data- Elements 
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Table 6. Summary of Empirical Data Log Elements 
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LIBRARY OF CONGRESS OHIO STATE UNIVERSITY 



SYRACUSE UNIVERSITY 





Collected 






Terminal ID 


1. 


2. 


System (SCORPIO or MUMS) 


2. 


3, 


File searched 


3. 


4. 


Comma nd 


4. 


5. 


Search key 


5, 


6. 


Begi ns 




7. 


Ends 


6. 



Not Collected 1 

Tine 

(Date) 

System response 

I of hits ' 

Patron Information 

Calculated 

State frequency distri- 
bution 

State chaining 

Chain length and patron 

frequency distributions 

Transition probability 

matrlceu 

Theoretical frequencies 
Sessions 

Session length In number 
of commands per aeaalon 



Collected f 

Terminal ID / 

Start time (enter coaiaand) 

Tine of system response 

Date 

System response (return 
cbde ) 

Search key text 
Observed 



1. 
2. 
3. 
4. 



9, 
10. 

11. 



Patron start time 

Sex of patron 

Patron or staff 

Help received from staff 

Not Collected 
Number of matches 
retrieved by search 
User ID 

Session end (patron end 
time) 

Calculated 

Elapsed time between 
transactions "gap time" or 
"command time" 
Session length I.e., number 
of commands per session 
Partial "session" length 
in elapsed time 
"Sessions" 

Relative frequency of 
return codes 
Card vs. LCS use 
Percent of patrons obtain- 
ing help 

I errors/I commands for 
1th command in aggregate 
session 

Return code by command 
State frequency distri- 
butions 

Chain length and pattern 
frequency distribution 



Collected 
Terminal ID 
Date 

Actual command 
System Response 



DALLAS PUBLIC 
Collected 



5, 
6. 
7. 
8. 
9. 
10. 



1. 

2. 

3. 
5. 



Terminal ID snd type 
User commands snd system 

responses (psrtlally coded) 
Date 

Tlue (In minutes) 
M Psrtlsl N begin session 



Not Collecte d 
Begin Session 
End Session 
User ID 



Calculated 

State frequency distri- 
bution 

Stste chaining 

Chain length and pattern 

frequency distributions 

Transition probability 

matrices 

Theoretical frequencies 

Time/session 

Commands/session 

Number of hits/search type 

Error types (frequencies) 

Sessions 



1. 
2. 
3. 
4. 



1. 

2. 
3. 

4. 
5. 
6, 
7. 



Not Collected 
User ID 

Some user Inputs? 
All "begin** session 
Session end 



Calculated 
Frequency of search 
commands in s session 
Sessions 

Number of terminals used 
esch day 

Transition frequencies 
Chain length fc pattern 
Theoretical frequencies 
Session length (time) 
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= Z 
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2.2 DATA COLLECTION AND ANALYSIS { 

There were three methods of data collection employed: Survey questionnaires, 
focus group interviews, and transaction log analysis. The goal for each of 
these methods is reported in Table 7 along with the accomplishments. , The 
questionnaires data collection was accomplished over a three-month period from 
April through June 1982. This was not an ideal time for most of the academic 
institutions since it was near the end of the term, but the timing was 
essential in the development of a national survey. 

The questionnaire data was analyzed by the research team at UC/DLA using the 
SPSS (Statistical Package for the Social Sciences). The same analysis was 
performed with each of the library's data. A summary was done by "type of 
library 11 as researched by the five principal organizations. The values for 
the libraries studied by OCLC are related in Vol. II of this final report 
along with the aggregate val ues. 

The data collected in the focus group interviews was done according to a 
discussion guide provided in Vol. II, Table 5. A focus group interview is 
usually conducted with 4-10 people and two researchers. One of the 
researchers is the discussion leader and the other heflps the group process as 
needed and manages the audio-recording equipment. Once the interviews have 
been conducted, .the tapes are transcribed for analysis. The analysis consists 
of categorizing reactions from the different groups interviewed. The groups 
are divided into users, nonusers, and library staff, and then further 
sub-categorized by type of group, e.g., students, faculty, reference staff, 
etc. Confidential reports of the interviews were made to each of the 
participating libraries. The anonymity of each individual who took part in 
the interview was protected. User anonymity and interviewers 1 skills made.it 
possible to have very open and frank discussions with the users and nonusers 
of OPACs. The interviews usually lasted an hour or more. A total number of 
404 people participated. 
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Table 7. Data Collection Goals/Accomplishments 



Institutions 



' --...Focused 

Questionnaires •■•>- Group * 

G/A interviews 
er Nonuser \f' (?/A 



U6< 



Transaction 

Log = > 

Analysis 

G/A 



t — " 

The Ohio State University 1500/1247 300/282 ^0/10 



Syracuse University 
Li brary of Congress 

Dallas Public Library 
Mankato State University 

Iowa City Public 
OCLC Libraries 



1500/1339 300/294 



Conducted by the 
staff at. the Li brary 
of Congress 

1500/1081 300/299 



Conducted by J. 
Matthews & Associates 
Inc. 

1000/410 200/144 
1000/329 100/499 



10/10 



10/16 



10/10 



10/14 



10/10 
Not 

planned 



2 months/one 

-year— 



2 months/three- 
weeks 

2 months/four 
months 



2 months/two 
weeks 

Not planned 



Not planned 
Not planned 



2.3 FINDINGS 

The findings from this project are reported in all three volumes. A summary 
of the general findings from the questionnaire data and the focus group 
interviews follows. For a full report, please read Vol. II, "Online Catalog 
Use: Results of Surveys and Focus Group Interviews in Several Libraries. 
The summary results for the transaction log analysis portion of the study, are 
also given below. For a full understanding the reader is directed to Vol. I. 
This chapter provides an overview of this study, while the next chapter 
specifically addresses its findings^ 
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2.3.1 Summary from Volume I, Current Utilization Of, Online Catalogs: 
Transaction Log Analysis; by John E. Tolle 

A. OBJECTIVE AND GOALS 

Part I of this study is concerned with the analysis of the transaction log 
records of online catalogs. The objective of this part of the online public 
access catalog study was to discover the extent to which current online 
catalog features are used and patterns of that use. While examining the 
actual catalog use, at least two goals became obvious: to obtain a better 
understanding of the use of online catalogs and to refine the methodologies 
used in carrying out any transaction analysis study. 

The first goal, to obtain a better understanding of the actual online catalog 
use, may seem to be too large or vague, as the objective is contained within 
it. However, it is important to keep in mind that a valuable asset of the 
research is that it generates questions that were not previously considered. 
As such, it contributes to an understanding of the catalog use. This study 
has achieved this in r the sense that considerable understanding of catalog use 
has been obtained. There is of course more to do and investigate, but we are 
much closer to understanding how the online catalog is used. 

The second goal, to refine the methodologies, was certainly achieved. The use 
of transaction analysis—via development of transition probabili ty matrices, 
state definition/, state diagrams, and Markov analysis—to create chains of 
user patterns certainly offers promise. This transaction analysis methodology 
is still in the ^arly stages of refinement, however. Further ref inement wil 1 
occur as more systems are examined and more data is analyzed. 

The scope of the/ transaction analysis included collecting data from four 
different online catalog systems. Machine-readable log tapes, which contained 
the user transactions in various formats and codes, were received from the 
following institutions: Library of Congress, Syracuse University, Dallas 
Public Library, and The Ohio State University. In addition, data was 
collected on the searching of 76 OCLC public terminals at 53 different 
libriri^s (as defined by unique OCLC location symbols). By institution type, 
these 53 libraries included 5 public libraries, 5 federal or state libraries, 
and 43 academic libraries (including several medical, health, and law 
libraries). 



B. DATA COLLECTION 

For the other online catalog systems, we obtained transaction log tapes 
covering the following periods: 

Library of Congress - March to June 1982 

Syracuse University - April 12 to April 28, 1982 

Dallas Public Library - August 9 to August 21, 1982 
The Ohio State University - January to December 1981, March to May 1982, 

and November 1982 
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Originally we requested two months of data from each library, but the actual . 
data received ranged from two weeks to one full year. Using this data, 
samples were selected for analysis. For Syracuse University and Dallas Public 
Library, data was collected for two weeks, and included an examination of the 
representativeness of subsets of data. For Syracuse, some of the days were 
eliminated due to some mislabeled tapes and tMme consideration. For the 
Library of Congress, public use data was analyzed for the months of March 1982 
and June 1982. April 1982 contained staff use data, and May 1982 had a 
mixture of both staff and public use data. These months were not analyzed 
since we were interested solely in patron usage. Spring quarter 1981 was used 
for transaction analysis at The Ohio State University because session 
boundaries were defined by observers stationed at the catalog. 

Each online catalog system was different; therefore, the resulting transaction 
log tapes varied in content and format. These differences are quite 
extensive, and the data reduction, coding, and reformatting represented a 
major task within the project. It was estimated, after the fact, that in 
conducting any transaction log analysis, well over 50% of the effort inlabor 
and computer time (including programming, tape reduction, and reformatting) 
can be attributed to this phase. 

Specific data elements collected (where available) included the following: 
terminal identification, user commands and codes, system commands and codes, 
the search key, session beginning, session ending, times of user and system 
responses, search text, and dates. 

The above list represents an ideal rather than the reality, since no OPAC 
system studied had all of these elements as part of the information captured 
on the system logs and copied to tapes for analysis. In order to appreciate 
the differences among the systems studied, consider that Ohio State, Syracuse, 
and Dallas Public did not have any session determinates, i.e., session length 
(when a user begins a session arid then ends the session) was not captured on 
their systems. 

The available data included manually collected start time at Ohio State, 
partial start time at Dallas, and start and end indications at LC. However, 
even though LC had "theoretical" session boundary markers, there were no time 
stamps on the data for time of day, nor were there any stamps indicating the 
date. 

For Dallas, time was recorded to the nearest minute, which is insufficiently 
detailed for transaction analysis because patrons enter commands more often 
than once per minute. At Syracuse, an attempt was made to capture session 
beginnings by requesting certain user input, but this was often disregarded. 
The use of the Ohio State data from 1981 allowed session determination by 
coordination with observers who recorded the beginning of sessions. Since 
this was the only direct observation data, it was thought to be advantageous 
to make additional use of it even though it was originally collected for 
another study. 

Syracuse had no user codes that were precoded. We obtained the literal use 
codes from the screei and had to search through the tapes until we came across 
the codes. System messages were also extracted in this manner. 



,24 



38 



Report Number: 0CLC/0PR/RR-83/4 
1983 March 31 



C. METHODOLOGY 

The task of analyzing the transaction tapes was broken down into the following 
steps: 

Obtain the transaction tapes. 

Check the data for validity and format; then copy them. 
Condense the tapes by eliminating any data unnecessary to the analysis. 
Determine sessions where possible. 

Conduct analyses of nontransi tional type (i.e., determine time and/or 
commands per session, etc.). 
Build state maps using all codes. 

Run analysis, check for problems* and condense the mapping further if 
required. 

Build special maps and subsets of data. 
Run analysis again. 

Examine results for observations and conclusions. 



The particular methodology chosen for analysis consists of first defining 
state codes and then developing taxonomies of these state activity codes (both 
user and system codes) that are mutually exclusive and exhaustive. Next, 
state mappings are determined that allow these state codes to be mapped into 
user or system states. User and system transactions are mapped into activity 
states. This mapping depends upon the user or system code assigned (if there 
Js one) and also upon the context of use. The state codes represent search 
phases that the user is in at a given time. These were grouped according to 
function into one of eleven primary states, ^such as states representing the 
beginning of a session, display of bibliographic recqrcte, set creation (simple 
searches), help functions, combined functions, and errors that may occur. 

Numerous mappings are possible for the same OPAC system. There is no one 
"correct" mapping; however, there may be "incorrect" mappings. For each 
system, a number of different mappings were made and analysis performed. At 
times, the output indicated the need for modifications in the selected maps. 
In these instances, changes were incorporated. As such, the primary mapping 
serves as a structure that may be applicable in varying degrees to different 
online catalogs. 

An individual taxonomy was often further refined in a more detailed taxonomy. 
For example, specific display commands could be broken down into 
display-author, display-title, and display-subject. It is possible to break 
out specific error types if such error types are defined by "the system under 
study. Similarly, certain paging commands may be associated with some 
specific states, but not others. 

When assigning commands to specific states, the context of the search should 
be considered. That is, we should consider the purpose or context of its 
actual use. A given command may have more than one use in practice. 
Therefore, its intended use should be considered, in addition to the semantics 
of the situation, before mapping the code to a state. It may thus be possible 
to map a command code to more thart one state, depending on the context of its 
use. 



1. 
2. 
3. 
4. 
5. 

6. 
7. 

8. 
9. 
10. 
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An example of this is shown below, using search terminology from the Library 
of Congress system. The third column illustrates the state mapping that would 
occur for an "original" map, while the right column illustrates where the code 
would be mapped for a "context validated" map. BGNS represents a begin 
session code, BRWS represents a browse code, and DISP represents a display 
state, while ENDS represents an end session state. 

Here, the paging commands were originally mapped into a display state since 
they resulted in a new screen display. But, when considering the actual 
context of paging command use, they were being used as a browsing command (a 
specific bibliographic record was not being looked for in this example). 
Thus, these commands were mapped to a browse state (BRWS or BRW+, which means 
browse with explicit paging). 

SEARCH SEARCH STATE 



CODE Original Map Context-Validated Maps 

BGNS LCCC BGNS BGNS 

BRWS PSYCHOTHERAPY BRWS BRWS 

tf paging command DISP BRW+ or BRWS 

0 paging command DISP BRW+ or BRWS 

0 paging command DISP BRW+ or BRWS 

ENDS ENDS ENDS 



Once these state codes and mappings were determined, transition probability 
matrices and state diagrams were calculated and constructed. State transition 
matrices simply represent in matrix form the probability (or percentage) of 
going from an origin state to a destination state (e.g., the probabil ity of 
starting in a display state and ending in an error state or starting in an 
error state and ending in a help .state). The state diagrams then illustrate 
the frequencies (percentage) of starting in a given state and ending up in 
another state. c 

Such matrices and diagrams can be calculated for 1st, 2nd, 3rd, and higher 
orders, where orders are defined as the number of transitions out of the 
origin state. That is, a transition from display to display is 1st order, 
while a subject to subject to author transition is 2nd order. 



D. OBSERVATIONS AND CONCLUSIONS 

In attempting to draw conclusions and observations from the transaction 
analysis data, a number of questions were proposed and then addressed with 
regard to specific systems. A number of the original questions could not be 
answered because of the nature of the available data (e.g., time stamp not 
available on given systems) and data collection period. Observations and 
conclusions include: 

The amount of time spent at the terminals was not available with any accuracy 
except at The Ohio State University, where it. ranged from an average of 
approximately 4 minutes to over 9 minutes, depending on the library considered 
i.e. Engineering, Undergraduate, West Campus, Education or Main Library. In 
all cases, the time spent at terminals was higher than the time spent at card 
catalogs (if available) at the same institution. 
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Errors frequently occurred in sequence, i.e., when an error was made, there 
was a high tendency to make another error (.286 at Syracuse, .598 at LC, .333 
at 0SU). c At Dallas Public, few errors were made, a situation we attribute to 
the format-filled system, which allows for few syntactic errors. - 

A number of questions are raised by the tendency to remain in an error state, 
including: Would help commands be useful? Are the help commands available 
useful? 

The data indicates that, for at least three of the systems, Library of 
Congress, Dallas, and The Ohio State University, we are dealing with fairly 
homogeneous data. We cannot say with any certainty that we have a random 
sample, especially at Syracuse, Dallas Public, and Library of Congress. No 
significant difference was found in the Oth order probabilities for The Ohio 
State University when using spring quarter data. Similarly, the March and 
June data from LC were comparable, as were the 1st and 2nd week of the Dallas 
Public data. 

These comparisons were done on simple frequencies of command usage. It is not 
possible to compare state chaining patterns of higher orders with any degree 
of confidence. We are dealing with complex chains and behaviors that are 
dependent on numerous, undefined variables. Indeed, even with regression 
analysis techniques that are well defined, it is important to note that 
results are very dependent on the observed data sets. Another factor is that 
random samples were not obtained. The systems are not designed to provide all 
the required data or to select random samples for analysis. Random samples 
would be necessary for strict sampling validity. 

For LC, Syracuse, and Ohio State, the transitions of 1st and 2nd order between 
identical states have high probability, usually the 1st or 2nd highest 
probability in the transition matrices. This would seem to imply that users 
have a tendency to stay or return to the same state up to a chain of three 
commands. In addition, a detailed analysis of 2nd orders tends to confirm 
that they remain in the same state throughout the chain. 

Search states have a tendency to go to the same states. For example, an 
author search tends to proceed to another author search, a subject search 
tends to proceed to another subject search, etc. Users seem to remain in the 
same search state for a long time. As previously mentioned, this also occurs 
for the error state. 

A look at the most frequently used search types by title, author, subject, 
etc., yields the following: 



0S\i{%) 
Spring 1981 



DALLAS (%) 
April 1982 



SYRACUSE (%) 
April 1982 



Title 
Subject 
Author 
Author/ title 
Other 



30.8 
24.6 
16.8 
10.5 
17.3 



16.0 
60.0 
15.0 
3.6 
5.4 



20.7 
26.2 
•24.0 



3.7 (Boolean) 
25.4 
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Subject searches are highest at Dallas and Syracuse, but at Syracuse, where 
the subject search is the highest percentage search, more sessions begin with 
an author search than with a subject search (25.1% compared to 21.9%). This 
"enhanced" number of subject searches may be attributable to users who use the 
search often within a session and may not be an indication of the type of 
search chosen. In addition, subject searches result in zero hits more than 
other searches; with the exception of a DUAL search (Boolean) at Syracuse. 
This indicates that subject searches are not a particularly successful type of 
search. 



E. TRANSACTION ANALYSIS GOALS 

In summary, transaction log analysis is an effective tool for both formulating 
the questions concerning the actual user behavior at online catalogs and in 
answering these questions. There are a number of goals in studying this 
behavior, including the determination of how users perceive the system, how 
the system fits the users' needs, and^how the system is really being used. 
This third goal most obviously lends itself to transaction log analysis. The 
users 1 perception of the system and how it fits their needs helps to dictate 
how the system is actually used. 

In order to approach a transaction analysis, it is first necessary to consider 
the system at a "lower" zero level. Next considered are first order 
descriptions, very simple user behaviors and system responses (e.g., whether 
the user hits a certain key or another key, and whether the system responds 
with a certain message or another message). At this point, we start 
collapsing strings of units that are coupled with a high probability into more 
comprehensive behavior descriptions. For example, one could collapse all 
repeated search strings together, such as author-author-author or repeated 
error strings. Or, one could collapse into a specific ^search command state 
with hits and another search command without hits. At this stage, the 
analysis is rerun (i.e., recalibrated) with new coding/state mappings, and 
other highly coupled states are sought. 

Another use of the method of transaction analysis is to test hypotheses 
concerning user behavior at online catalogs, e.g., to test the hypothesis that 
a user tends to stay within the same search—author, subject, or title--wi thin 
the same session. Other hypotheses may look at error occurrences and search 
frequencies. 

Transaction log analysis may also be utilized to conduct basic exploratory 
data efforts and to generate questions that we may or may not be aware of. An 
example of this is the finding that the highest frequency of a search type is 
author search, while subject search is 2nd highest (this is only an example). 
Examination of the data may reveal, however, that within sessions multiple use 
of the author search is used much more frequently. Thus, when examining the 
frequencies of sessions that have specific search types, subject search may be 
highest. 
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This first online access catalog transaction log analysis study has helped to 
refine the goals of transaction analysis. It has also revealed some of the 
problems associated with conducting such a study which should be beneficial to 
future efforts. It has increased our knowledge of how these systems are 
actually being used and provides data and results with which to formulate 
questions and conclusions on the operation of OPACs. As such, it offers an 
opportunity to build upon these efforts. Transaction log analysis offers an 
opportunity to expand our knowledge of the actual use of OPAC systems. 

2.3.2 Summary from Volume II, Online Catalog Use: Results of Surveys and 
Focus Group Interviews in Several Libraries, by Karen Markey 

A. PROJECT OBJECTIVE 

This report covers the analysis of questionnaires and focus group interviews 
at several libraries using online public access catalogs. The objective of 
the research project was to evaluate online public access catalogs (OPACs) 
through an analysis of patrons' needs and perceptions. The evaluation of 
existing OPACs is necessary to ensure that present and future development of 
such catalogs will lead to greater acceptance by library patrons. 



| B. METHODOLOGY 

i 

Self-administered questionnaires for users and nonusers of OPACs were 
constructed by a committee composed of the principals whose studies of OPACs 
were sponsored by the Council on Library Resources. The survey instruments 
were pretested formally and informally prior to the actual study. In spring 
1982, 29 institutions took part in the administration of user and nonuser 
i questionnaires. OCLC was responsible for questionnaire administration at the 
following 10 institutions: 

! 

1. University of Akron 

2. Case Western Reserve University 

3. Dallas Public Library 

4. Iowa City Public Library 

5. The Ohio State University 

6. Ohio University 

7. State Library of Ohio 

8. Syracuse University 

9.. University of Texas at Austin 
10. University of Texas at Dallas 

The user questionnaire contained 59 questions, which were organized into the 
following four parts and subject areas: (1) most recent OPAC search; (2) 
experiences with OPAC features and services; (3) improvements to the OPAC; and 
(4) demographic information about the OPAC user. This questionnaire was 
distributed to "users" of OPACs who -were defined as individuals 14 years old 
or older observed using the OPAC, i.e., sitting or standing in front of a 
terminal and touching keys or the screen, looking at a display device, etc. 

The nonuser questionnaire contained 15 questions, which were organized into 
the following parts: (1) expectations of the OPAC; and (2) demographic 
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information about the OPAC nonuser. This questionnaire was distributed to 
"nonusers" of OPACs who were defined, as individuals 14 years old or over who 
responded negatively to the data collector's question asking whether they had 
ever used the library system's OPAC. 

The time blocks when users and nonusers were recruited to complete a 
questionnaire were randomly selected. The sampling plan was not the same from 
library to library but took into account each institution's calendar, hpurs, 
and special events. Data collection at Case Western Reserve University did 
not take place until fall 1982 and winter 1983; the results of Case Western's 
questionnaire administration are given in Vol. II, Appendix D. At eight of 
nine surveyed libraries, the acceptance rate of users exceeded 60%. The 
acceptance rate of nonusers equaled or exceeded ^65% at all nine surveyed 
libraries. 

Focus group interviews were used to collect information on library staff and 
patron needs and perceptions of online catalogs. Four groups of participants 
were sought for focus group interviews: (1) library patron users of OPACs; 1 
(2) library patron nonusers of OPACs; (3) reference services library staff; 
and (4) technical services library staff. Focus group interviews were 
conducted at the following six institutions: 

1. Dallas Public Library 

2. Iowa City Public Library 

3. Library of Congress 

4. Mankato, State University 

5. The Ohio State University 

6. Syracuse University 

The analysis of questionnaire data was performed at UC/DLA using SPSS to 
produce descriptive and nonparametric statistics. Content analysis of 
discussions in focus group interviews was used to construct a descriptive 
framework in which participants' remarks were- categorized into four general 
areas: (1) existing system; (2) card to computer catalog transition; (3) 
database; and (4) user assistance (see Vol. II, Table 7). Comments of 
interview participants were used in this report to provide insight into and 
assistance in interpreting quantitative questionnaire results. Focus group 
interviews were also an important source of comments from groups of OPAC users 
and nonusers who were not surveyed because they did not match the definition 
of a user or a nonuser, and from members of relatively small but vocal groups, 
e.g., university faculty, older adults, etc., whose survey responses were 
overshadowed because they were represented in the population in relatively 
small groups in comparison with the majority of respondents. 

C. DEMOGRAPHIC BACKGROUND OF SURVEYED USERS AND NONUSERS 

1. Frequency of Library Visits by Library Patrons 

• Differences in frequency of library visits were found between library 
patrons and the types of libraries they visited and between OPAC 
users and nonusers. Library patrons at public libraries and federal 
and state libraries visited the library less often than patrons at 
academic libraries (see Vol. II, Table 9). 
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• Library patrons at Syracuse, Ohio State, and OCLC-system libraries 
(i.e., University of Akron, Ohio University, State Library of Ohio, 
University of Texas at Austin, and University of Texas at Dallas) 
visited the library on a daily or weekly basis. Library patrons at 
the public libraries of Iowa City and Dallas visited the library on a 
weekly or monthly basis. OPAC nonusers at these five surveyed sites 
visited the library less frequently than users (see Vol. II, Table 
10). 

Frequency of OPAC Use 

• Differences in frequency of OPAC use were found between patrons and 
the types of libraries they visited. Library patrons at ARL -member, 
academic, and community college libraries used the OPAC less 
frequently than public library patrons. A large percentage of 
first-time OPAC users were found at federal and state libraries (see 
Vol. II, Table 11). 

• Library patrons at the public libraries of. Dallas and Iowa City used 
the OPAC on every or almost every library visit. Patrons at Syracuse 
and Ohio State used the OPAC on almost every library visit or 
occasionally. OCLC users consulted the system less frequently than 
OPAC users at the four other surveyed libraries. Most likely, OCLC 
users consulted the system only when they could not find an item in 
the library's local catalog (see Vol. II, Table 12). 

Age of Surveyed Respondents 

• Differences in ages of surveyed respondents were found between 
library patrons and the types of libraries in which they were 
surveyed. Most surveyed users and nonusers represented the 15 to 19, 

. 20 to 24, and 25 to 34 age groups (see Vol. II, Table 13). 

• The ages of OPAC users and nonusers were in similar proportions in 
each age group, except for the high percentage of older adult public 
library nonusers {15%) in comparison with the corresponding low 
percentage of older adult public library users (4%), and the high 
percentage of young adult public library users (27%) in comparison 
with the corresponding low percentage of young adult public library 
nonusers (12%). Older adults in focus group interviews brought to 
light their intimidation by the OPAC and their perception of 
youngsters' flexibility, eagerness, and receptivity toward the OPAC. 

Sex and OPAC Use-Nonuse 

• With respect to the CLR-aggregate data (i.e., data collected at the 
29 participating libraries), a significant relationship was found 
between sex and OPAC use-nonuse. In general 1 , a greater percentage of 
users was male (see Vol. II, Table 15). 
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§ With respect to the five OCLC-surveyed sites, a significant 

relationship was found between sex and OPAC use-nonuse at only Dallas 
Public Library. Statistics from Syracuse, Ohio State, Dallas Public, 
and OCLC-system libraries indicated that there were more female than 
male first-time users (see Vol. II, Table 16). Should such a trend 
toward female first-time use continue, percentages of male and female 
users might equalize in the near future. 



5. Academic Background of Survey Respondents 

§ Percentages for the highest grade completed by survey respondents 
showed little difference between users and nonusers at individual 
libraries or types of libraries. The majority of academic library 
patrons were in college; public library and federal and state library 
patrons were college graduates (see Vol. II, Table 17). 

• With respect to the five OCLC-surveyed sites, the highest grade 

completed by survey respondents reflected findings in the aggregate 
grouping by type of library. The majority of library patrons at the 
academic libraries of Syracuse and Ohio State were undergraduates. 
At the public libraries of Dallas and Iowa City, the largest 
proportion of library patrons were college graduates (see Vol. II, 
■ Table 18). 



D. SEARCHING THE ONLINE PUBLIC ACCESS CATALOG 
1. Purpose of OPAC Searches 

• Differences were found in the analysis of aggregate data between the 
purposes of users' OPAC searches and the type of library at which 
they had performed their searches. At ARL -member, academic, and 
community college libraries, OPAC users' purposes were class-related. 
Personal interest was the chief purpose of public library searchers. 
OPAC users at federal and state libraries searched the OPAC for a 
variety of purposes (see Vol. II, Table 21). 

• Individual libraries' statistics on OPAC users' search purposes 
reflected findings of the aggregate data except the purposes of 
OCLC-system users of which a large percentage was advanced research 
(see Vol. II, Table 22). 

• The majority of Syracuse, Ohio State, and OCLC-system users who were 
undergraduates were involved in searches for class-related purposes. 
A large percentage of graduate student users of the Syracuse and Ohio 
State OPACs were also performing searches for cl ass^related purposes. 
Faculty members' search purposes were related to their jobs or 
research (see Vol. II, Table 23). 

• ' The main focus of OPAC users' academic work was course work, except 

at OCLC-system libraries, where both research and course work were 
the main foci (see Vol. II, Table 24). 
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2. Type of Information Entered into the Computer Catalog 

• Multiple responses to questions about the type of information users 
brought to and entered into the computer catalog and the type of 
materials they were trying to find confounded the results. Rather 

than risking erroneous interpretations from such confounded data, two 
methods are recommended for collecting more reliable data in future 
studies: (1). online questionnaire administration after the 
completion of an OPAC search that can be captured in a transaction 
log; and (2) "offline" questionnaire administration after OPAC 
searchers' expressed thoughts are captured through protocol analysis. 



3. Finding the Object of the OPAC Search 

• Fewer than 20% of OPAC searchers did not find any of the desired 
material. The majority of OPAC users found at least "some of the 
items for which they were looking (see Vol. II, Table 26). 

• More very infrequent and first-time users found nothing as a result 
of their search than frequent OPAC users (see Vol. II, Table 27). 
This finding: (1) suggests that lack of proficiency in the use of 
OPACs may adversely affedt retrievals; (2) supports the requirement 
that OPACs should contain features that contribute to the ease of 
rel earning the system; and (3) suggests that OPACs contain features 
that assist online searchers with alternate search strategies when 
their searches retrieve ijiothing. 

• In general, one of every (two OPAC searchers encountered other things 
of interest in the coursjj of their online search (see Vol. II, Table 
28). Finding other things of interest was related to personal 
interest searches, a purpose characteristic of public library OPAC 
searchers (see Vol. II, Table 29). 



4. Use of the Library's Traditional Catalog * 

• Use of the library's traditional catalog by library patrons was 

connected to: (1) the capability of the online system, e.g., urn on ( 
catalog, local library Catalog, etc.; (2) the purpose of the patron s 
search; (3) the comprehensiveness of the online system s coverage 
with respect to the traditional catalog's coverage; and (4) 
combinations thereof. Should any of the first three factors change, 
library patrons' use of | the traditional catalog would probably be 
affected. 



E. TRANSITION FROM TRADITIONAL LIBRARY CATALOGS TO OPACS 
1. General Attitudes toward OPACs 

• Over 80% of surveyed users and nonusers had favorable attitudes 
toward the OPAC. Nonusers were less exuberant than users in their 
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expression of favorabil ity (see Vol. II, Table 31). Comparable but 
different studies of OPACs also reported favorable responses by users 
and/or nonusers about OPACs. 

At this early stage in the development of computer catalogs, few 
users have had the opportunity to use more than one OPAC. Only about 
25% of surveyed library patrons were frequent users of other computer 
systems (see Vol. II, Table 68). Favorable attitudes of library 
patrons might change in time as they have more familiarity and 
practice" with other OPACs and computer systems and have greater 
powers of discrimination. 



2. Favorable Attitudes toward the OPAC 

• The four following reasons for favorable attitudes toward the OPAC 
were expressed in focus group interviews: 

a. Searching the computer catalog is fun. Especially among 

youngsters, young adults, and university students, searching the 
OPAC was more fun and less boring than searching the traditional 
1 ibrary catalog. - 

\ b. Searching the OPAC saves time. The computer catalog saves time 
in a number of ways: (1) ability to view the same information 
that another searcher is viewing; (2) instantaneous response of 
the OPAC in comparison to flipping catalog cards; (3) ability to 
remain in one place and search many different access points; (4) 
ability to access the library's catalog from remote locations, 
e.g., campus terminals, home terminal s, etc. ; (5) availability 
of circulation information. 

c. The OPAC provides new services. Examples of these new services 
are printouts of retrieved citations, ability to search the 
catalogs of other branch or campus libraries, and mail delivery 
of books to home or office. 

d. The OPAC provides new features. Examples of these new features 
are author/title search, limit command, and component word 
searching of subject-rich MARC fields. 

• The three following variables were related to OPAC users' attitudes 
in analysis of survey questions: 

a. Finding other things of interest during the OPAC search was 
related to OPAC users' general attitudes toward the OPAC (see 
Vol. II, Table 32). For example, the percentage of users who 
did not find other things of interest and were unfavorable 
toward the OPAC was over three times greater than the percentage 
of users who were unfavorable toward the OPAC and did find other 
things of interest. 

b. Users' attitudes toward the library's other catalogs were 
related to their general attitudes' toward the OPAC (see Vol. II, 
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Table 33). For example, OPAC users who felt thatthe OPAC was 
Worse than the library's traditional catalog were also 
unfavorable toward the OPAC. 

c. Frequency of OPAC use was related to users' general attitudes 
toward the OPAC (see Vol. II, Table 34). For example, only Q% 
of frequent users were unfavorable toward the OPAC, whereas 21$ 
of infrequent users were unfavorable. 



3. Unfavorable Attitudes toward the OPAC 

• The three primary reasons why surveyed nonusers had not yet used the 
OPAC were: (1) not having taken training sessions on how to use it; 
(2) not having time to learn how to use it; and (3) no need to use 
any 1 ibrary catalog (see Vol. II, Table 35). Two reasons for nonuse 
that emerged in focus group interviews, i.e., "card catalog is easier 
to use" and "dislike of computers", were not highly ranked among 
surveyed nonusers but were reasons selected by nonusers who were very 
unlikely to use the OPAC in the future (see Vol. II, Table 36). 

• The five following reasons for unfavorable attitudes toward the OPAC 
were expressed in focus group-interviews: 

a. Library patrons were familiar with and trusted the traditional 
library catalog. Library staff could help build patrons' trust 
in the OPAC by making clear to patrons: (1) why the library has 
chosen to convert to the OPAC; (2) comparison of the traditional 
catalog's contents with" the computer catalog's contents; (3) 
types of assistance available for learning how to use the OPAC; 
and (4) benefits of the OPAC. Such information is important to 
disseminate before and after the introduction of the OPAC. 

b. Library patrons feared computers or mechanical devices in 
general. Their fear may dissipate as the use of computers in 
everyday T iving becomes commonplace. In the meantime, a variety 
of online and offline user aids is suggested. 

c. Library patrons were reluctant to learn how to use the OPAC. 
Some of these patrons were merely "burned out" on learning, such 
as university students who viewed the OPAC as just one more 
aspect of university life that had to be learned and mastered. 
Other patrons, especially older adults and faculty members, were 
embarrassed about their lack of knowledge, which intensified 
when they compared their skills with those of youngsters or 
undergraduates. "Burned out" patrons will probably learn how to 
use the OPAC at the last minute before an assignment's deadline; 
consequently, workshops or individualized instruction must 
always be available, not just at fall orientation or in the 
first few months of the OPAC's residence in the library. 
Self-conscious or embarrassed patrons will probably have to be 
approached on an individual basis by library staff. Peer 
teaching by fellow faculty members was also suggested as a means 
of assisting reluctant facul ty members. 
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t Statistical relationships were found between nonusers' attitudes 

toward the computer catalog and: (1) perceived length of time needed 
to learn to use the OPAC (see Vol. II, Table 37); (2) perceived 
difficulty of learning to use the OPAC (see Vol. II, Table 38); and 
(3) likelihood of future OPAC use (see Vol. II, Table 39). 



4. Satisfaction with the Most Recent OPAC Search 

• Over 75% of users overall and at indivi dual libraries were satisfied 
with th* results of their most recent OPAC search. (see Vol. II, Table 

40) . 

t Searchers who discovered other things of interest during their most 
recent search expressed greater degrees of satisfaction than those 
who had not discovered other things of interest (see Vol. II, Table 

41) . 

• Satisfaction with the most recent OPAC search was related to OPAC 
users' comparisons of the OPAC with the library's other catalogs (see 
Vol, II, Table 42). OPAC users who were not satisfied with their 
search generally felt that the OPAC was worse than the library's 
other catalogs. 



5. Changes in OPAC Users' Search Habits 

t The introduction of the OPAC to the library resulted in the following 
changes to the search habits of OPAC users, which were voiced by 
library patrons and staff in focus group interviews: 

a. Library patrons enjoyed searching the computer catalog more than 
the traditional library catalog. 

b. Library patrons took advantage of new features and services of 
OPACs. 

c. Library patrons noted that the ability to remain in one place 
and check multiple access points increased their perseverance. 

• Library patrons who noted that their search habits had not changed 
were performing bookshel f browsing searches in the computer catalog. 



F. FEATURES OF ONLINE PUBLIC ACCESS CATALOGS 

1. Review of Features in OPACs 

t OPAC features were reviewed in system-by-system checklists for 

operational control, search formulation control, output control, and 
user assistance (see Vol. II, Tables 45-49). 

t When checklists of OPAC features were matched with the user 

questionnaire, it was demonstrated that: (1) there were features 
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enumerated in the checklists that none of the systems in the study 
offered; (2) there were features in the systems studied that were not 
evaluated because the questionnaire did not contain questions about 
those features; (3) there was not a one-to-one correspondence between 
survey questions and OPAC features enumerated in checklists; and (4) 
survey questions about users 1 experiences with features were 
closed-ended. 

• Focus group interviews provided a rich source for learning the nature 
of users 1 problems with OPAC features, helping to interpret survey 
results with regard to system features in general and disclosing 
users 1 problems with OPAC features that were not tested by the 
questionnai re. 



2. Users 1 Favorable Experiences with OPAC Features (see Vol. II, Table 50) 

• Search Formulation Control Features 

a. Users reported favorable experiences in connection with access 
points for known-item searching. The entry and amount of 
known-item information differed widely from system to system; 
however, subject access points were not listed among features 
with which users had favorable experiences, even though entry 
techniques were similar to known-item entry techniques. Thus, 
OPAC users 1 favorable experiences about known-item access points 
might be attributed to the concreteness of the task of finding 
known-items in comparison to the increased amount of mental 
effort involved in the task of subject searching. 

b. Focus group participants noted that the virtue of the 
author/title access point was its ability to pinpoint the needed 
item with a minimum of screen manipulations. . 

c. Ohio State OPAC users cited the helpfulness of the shelf 
position search (SPS) to complete subject searches and/or browse 
the shelflist. 

• Output Control Features 

a. Bibliographic and circulation record displays of the five 
systems studied in the survey were featured in a discussion of 
the ease of understanding single bibliographic record displays 
(see Vol. II, Figures 1-6). Individual lines of text in single 
record displays of the four systems with which users noted 
favorable experiences were prefaced by alphabetic and/or numeric 
labels. The use of such labels may set off lines of text to 
indicate to searchers differences in meaning from one labeled 
line of text to another. 

b. Circulation and bibliographic record displays were the chief 
locations of abbreviations, codes, undefined messages, jargon, 
etc. Focus group participants noted difficulty understanding 
record displays with such information. Displays and command 
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names should be reviewed by library staff and potential OPAC 
users, or be cpded in such a way that would easily accommodate 
changes after the introduction of the OPAC to the library (see 
Vol . II,. Figure 6). 

c. Recommendations for the placement of call numbers in 
bibliographic record displays are: (1) placement aTong the 
outer edge of text; (2.) far from the neighborhood of other 
numbers and characters that could be confused with call numbers; 
(3) labeling call numbers with a label understandable to 
patrons; and (4) explanations for and examples of circulation 
and bibliographic record splays in offline and on-line- user 
aids ^seeJtoW-WrFlgures 10-12). 

d. Iowa City Public's OPAC users favorably cited the memorability 
of the order of entering commands or the memorability of a 
particular command in the course of the search. The OPAC's 
horizontal array of touch commands might have helped patrons to 
remember the correct sequence of commands or the correct command 
(see Vol. II, Figure 13). Yet, Iowa City patrons mentioned the 
need for more directive means of informing them of the next and 
best step in the ongoing search. 

• Online and Offline User Assistance 

a. Syracuse University and Iowa City Public OPAC survey respondents 
cited their system's online user assistance. The Syracuse 
OPAC's directive prompts following the report of the number of 
retrievals helped searchers in the display of citations (see 
Vol. II, Figure 14). Help screens were available at all times 
in the Iowa City Public OPAC, but staff realized that users were 
embarrassed to be seen using them. The library's experience 
with changing the initial instructions after finding out 
patrons' reactions demonstrated the necessity for allowing 
individual libraries to alter screen displays, help messages, 
command names, field names, etc., after implementation of the 
computer catalog (see Vol. II, Figure 15). 

b. OPAC users were very favorable toward librarian assistance. 
OCLC users and OPAC users at Ohio State noted the helpfulness of 
brochures about OPAC use, and Ohio State users noted that 
brochures were always available. Ohio State was the only 
library surveyed where there were always free terminals 
available. 



3. Users' Unfavorable Experiences with OPAC Features (see Vol. II, Table 53) 

t Subject Access 

a. Analysis of focus group remarks and questionnaire responses 
showed that OPAC users experienced difficulty when performing 
subject searches. Reasons underlying this difficulty were: (1) 
patrons' inability. to match their input term(s).with the OPAC's 



52 



Report Number: 0CLC/0PR/RR-83/4 
1983 March 31 



controlled vocabulary; (&) the burden on the OPAC user of 
conjuring up broader or narrower terms; and (3) problems 
consulting the printed volumes of LCSH during the online search. 

b. The following six online user aids were discussed and examples 
provided that were intended to help patrons select searching 
vocabulary, browse the OPAC, and make satisfactory relevance 
assessments about displayed items: 

1. Alphabetical displays of subject headings (see Vol. II, 
Figure 16). Focus group participants pointed out that this 
feature has been present in searching the traditional 
library catalog. 

2. Keyword-in-context subject heading displays (see Vol. II, 
Figure 17). Such displays would help searchers match their 
entry vocabulary with the catalog's controlled vocabulary 
regardless of the position of the matched term(s) in a 
precoordinated (or bound) subject heading. 

3. Online^d.i splays of related subject headings (see Vol. II, 
Figure 18 and Table 54). Displays of related subject 
headings were the top-ranked additional OPAC feature in the 
survey. 

4. Augmented subject access to bibliographic records. 
Augmented subject access to records was ranked second in 
the list of additional OPAC features. Of Atherton's four 
benefits derived from a study of augmenting bfbl iographic 
records, "the ability to answer some queries impossible 
using today's catalog information" was what focus group 
participants had in mind when they requested "subject 
augmentation to bibliographic records (Atherton, 1978, p. 

Mv). 

5. Automatic linking of input free text terms to the OPAC's 
controTTed vocabulary. This is already operational in the 
CITE and PaperChase OPACs and helps searchers to improve 
the recall of subject searches. 

6. Inclusion of subject headings in bibliographic record 
displays (see Vol. II, Figure 19). This would be the 
alternative to mechanizing the procedure in 5. above. 

Call Number or' Shelflist Searching 

a. OPAC users had unfavorable experiences with the call number or 
shelflist displays in the Iowa City Public OPAC. Since such 
displays were probably used to find out if library materials 
were available in the library, the call number search could be 
simplified to reduce the number of steps (or touch commands) 
that are necessary to check 1 ibrary material s' circulation 
status (see Vol. II, Figures 20 and 21). 
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b. Ohio State users reported favorable experiences with shelflist 
searches and displays. Since bibliographic records entered into 
the Ohio State OPAC prior to June. 1978 lacked subject headings, 
searchers used the shelf position search to improve the recall 
of OPAC subject searches by using the results of a subject 
heading search to identify a fruitful call number area. The 
shelf position search also allowed searchers to "browse the 
stacks" by perusing titles of books in the area of the input 
call number. 

c. Focus group interview participants at all six libraries 
recognized the usefulness of browsing books in the neighborhood 
of a relevant book. Three suggestions are given to take 
advantage of the classifed arrangement of books: 

!• Automatic feedback routine that uses class number areas of 
relevant retrievals to rind additional items. ThisTouTTne 
nas already been implemented in CITE, a prototype OPAC at 
the National Library of Medicine. 

2. Online Subject Guide. This reports books' class number 
areas, as well as the number of books found in each class 
number area under the subject heading of interest (see Vol. 
II, Figure 22). 

3. Display of classification captions enumerated in 
classification schedules to help searchers find books on 
specific, coordinate ; or general topics (see vol. II, 
Figure 23). The need to browse subjects in context was 
also expressed by survey respondents when they ranked "list 
of related words" first among additional OPAC features. 

• Reducing or Increasing Search Output 

a. Questionnaire results concerning unfavorable features pointed 
out OPAC users' problems with reducing or increasing search 
output. Analysis of transaction logs (see Volume I) and focus 
group interviews showed that reducing or increasing search 
output is a widespread and prevalent problem. 

b. The following recommendations are made to help searchers reduce 
the number cf retrievals in highly posted known-item searches: 

Suggestive or di recti ve prompts to inform searchers to try 
the search using the author/title approach. In section ". 
b.l.l (see vol. II), searchers championed the author/title 
approach as a means of pinpointing the needed item. 

2 « Suggestive or directive prompts to inform searchers to 

limit results by various criteria. Analysis of transacti on 
data (see Volume I) showed that few searchers used limit 
commands, which suggests that they need to be prompted to 
use them. ; 
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c. The following are recommendations regarding assistance to OPAC 
searchers whose known-item searches result in no retrievals: 

1. If the author or title approach results in no retrievals, 
the system produces suggestive or directive prompts 
informing the user to retry the search using a title or 
author approach , . respectively. 

'2. When an author/title' search results in no retrievals, the 
system suggests or directs the user to enter only title or 
only, author information. 

3. Author/title keyword searches. The Syracuse OPAC already 
allows author/title keyword searches. Searchers need to be 
prompted when to enter them. 

4. Automatic truncation of title word(s) and/o^ author names. 
CITE, a prototype OPAC of the National Library of Medicine, 

* performs automatic stemming routines on users 1 input words 
and names. 

5. ' Spelling correction routines. Spelling correction routines 

have been implemented in CITE at the National Library of 
Medicine and BACS at the University of Washington School of 
Medicine. 

6. Name authority control capability. The University of 
California's system is an example of an OPAC with name 
authority control . 

d. Recommendations were given to help searchers reduce the output 
of highly posted subject searches: 

1. Suggestive prompts informing users that results can be 
reduced by the entry of Boolean operators and additfonal 
searchable information, limit criteria, or subject 
subdivisions, etcT ! " ■ / ~ 

2. Display of related word lists to help users find more 
specific or additional Boolean operators. The relatively 
infrequent use of advanced features such, as Boolean 
operators (see Volume I) indicates the need for prompts. 

3. Online Subject Guide consisting of displays enumerating the 
class number areas of books, as well as the numbei of books 
found in each class number area under the subject heading 
of interest. Searchers, especially bookshelf browsers, 
would be directed to the areas of the bookshelves where the 
majority of books under that topic are located (see Vol. 
II, Figure 22). 

e. Recommendations for assisting OPAC users whose subject searches 
result in no retrievals are: 



ERLC 



55 



41 



Report Number: 0CLC/0PR/RR-83/4 
1983 March 31 



1. Cross-references online. 

2. Related word lists to lead users to more general term(s). 

3. Automatic truncation of users 1 input subject term(s). 

4. Augmented subject access to bibliographic records. 

5. Automatic linking of input free text terms to the OPAC 1 s 
controlled vocabulary. 

t Output Control Features 

a. Difficulties with output control features occurred primarily in 
connection with entering commands. Traditional library catalog 
searching required a combination of mental exercise and physical 
skills. In OPAC searching, physical skills are "logical i zed" 
into a series of command names or touches. OPAC searchers must 
always keep in mind: (1) the correct command name; (2) correct 
command entry procedures; (3) correct information about the 
display, e.g., field labels, next record number to be displayed; 
and (4) appropriate times to enter commands. 

b. Comments in focus group interviews about command names, entry 
procedures, and the logical sequence of commands in online 
searches resulted in the following factors that must be 
considered in the design of output control features: 

1. Mnemonic command names or abbreviations. 

2. Command names or abbreviations that are easily 
understandable in view of the operation they perform. 

3. Command entry procedures that are easy to remember. 

4. Command entry procedures that are consistent from one 
command to the next. 

5. Suggestive or directive prompts that inform users of or 
guide them through possible actions. 

c. Problems with response time were usually specific to individual 
systems. Focus group interview participants were not 
particularly bothered by, slow response times; when they 
anticipated slow response times, they sought ways to beat them. 

d. The following output control features and the guidelines for 
them were drawn from an analysis and understanding of focus 
group participants 1 problems with specific OPACs 1 features: 

1. Display forward (of single or multiple bibliographic 
records). 
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2. Display backward (of single or multiple bibliographic 
records). 

3. Report of the number of retrieved citations or screens of 
titles resulting from the input of an access point. 

4. Report of the number of screens required to display a full 
bibliographic record and/or circulation information in the 
first screen of the record, which: 

a. Includes a very visible and understandable directive 
prompt at the end of the display to guide users to 
successive screens of the record, 

b. In the case of multiple, i.e., 5 or more, screens for 
a displayed record, includes an informative and 
understandable directory to the contents of multiple 
screens. 

5. Use of messages regarding circulation status that the 
library can live up to. 

6. Display of output ordered by a characteristic that is 
easily and quickly understood by users. 

7; Inclusion of subject headings in brief or intermediate 

level displays of citations retrieved in subject searches. 

• Operational and Search Formulation Control Features 

a. Users' difficulty interrupting the display of information and 
response time associated. with displaying the next bibliographic / 
record from a screen of records resulted in the description of a ; 
"report" key that would allow users to obtain a report of the 
system's progress thus far. 

b. Users' unfavorable experiences with truncation led to the two 
following recommendations: 

1. Automatic truncation of users' input access points when few 
or no records are retrieved. 

2. Natural language input of access points and system 
construction of derived search' keys. 

• Offline User Assistance 

a. At all five libraries surveyed, users reported difficulty 
knowing what was included in the OPAC. Following are 
recommendations for informing patrons about the coverage and 
contents of the OPAC: 

1. Workshops, group or individualized instruction, and 
training materials used in workshops or instruction 
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sessions must include a discussion of the coverage and 
contents of the library's OPAC. 

2. Permanent printed guides or brochures about OPAC use to 
which searchers can refer when performing online searches 
at the OPAC, laminated and/or pasted to the terminal or 
terminal table, posted on walls, etc. 

3. Free printed guides or brochures about OPAC use that are 
available for the taking. 

4. Permanent signs posted near the liorary's traditional 
catalog must refer patrons to the OPAC to ensure that the 
results of their searches are up r to-date and comprehensive. 
A sign could say, "Check the computer catalog for books, 
government documents, magazines, and newspapers published 
after 1968." 

5. Online message at the end of OPAC search could inform users 
that there are other resources to check; for example, an 
online message might say, "Check the card catalog for 
books, government documents, magazines, and newspapers 
published before 1968." 

b. Excessive queuing at computer catalog terminals resulted in 
comments from focus group participants about the privacy of 
their online searches. The formation of a single queue where 
users stand to wait for the next available terminal is 
recommended to segregate patrons actually performing online 
searches from those waiting to perform them. 

c. Focus group interviews revealed sets of patrons who refused to 
obtain a librarian's assistance; wanted to teach themselves how 
to use devices on their own time and at their own convenience; 
or felt embarrassed or self-conscious about referring to a 
brochure or manual during their online search when other library 
patrons were around. Such patrons would benefit from available 
pamphlets, brochures, or instruction sheets that they could take 
with them and study at their own convenience. The 
implementation of a number of methods of teaching patrons how to 
use the OPAC is very important to accommodate the variety of 
personal learning styles and preferences of library patrons. 

d. OPAC users at four of the five libraries in the survey were 
eager for more computer catalog terminals in the library, in 
locations other than the library, and in library locations other 
than near the traditional catalog (see Vol. II, Table 55). 
Focus group interview participants were very keen on obtaining 
access to OPACs at home, in university offices , campus 
buildings, and elsewhere. 



58 



Report Number: 0CLC/0PR/RR-83/4 
1983 March 31 



OPAC Workstation Design Considerations 

a. OPAC users expressed problems with different types of terminals 
requiring different entry procedures. 

b. OPAC users' unfavorable experiences with amount of writing space 
and height of terminals suggest that libraries consider 
increasing writing space and providing chairs for OPAC 
searchers. 

c. Interview participants who were most concerned about comfort at 
computer catalog workstations were technical services and 
processing staff whose jobs required many continuous hours at a 
termi nal . 

d. Few of the libraries at which surveys or interviews were 
completed made hardcopy printers or printed output available. 
Survey respondents ranked printers high in a list of the 
top-ranked additional OPAC features, and focus group 
participants were very interested in obtaining hardcopy 
printouts of retrieved citations. 

e. Other services and features expressed in a number of group 
interviews were: 

1. Same-day service for hardcopy printouts. 

2. Placement of an automatic hold on retrieved materials in 
circulation. 

3. Automatic delivery of library materials to patron's home or 
office. 

4. Designation of the location where a retrieved item is 
shelved. 

Enriching the OPAC' s Database 

a Journal titles were ranked first by surveyed patrons at academic 
and public libraries amoung library materials that should be 
added to the OPAC (see Vol. II, Table 56). Interview remarks 
demonstrated that users were referring to titles of journal 
articles in individual issues. Thus, patrons were interested in 
the addition to the OPAC of the contents of periodical indexes 
such as Rea der's Guide to Periodical Literature , General 
Literature" Index , and of specific subject-on entedpenodical 
indexes, e.g., social sciences, humanities, etc. 

b. Newspaper articles, subject bibliographies, and other library 
reference works were mentioned by survey respondents and/or 
interview participants as materials they would like added to the 
OPAC Patrons' interest in the addition of important reference 
literature to the OPAC's database suggests the implementation of 
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the following two OPAC features which employ the data already 
available in the MARC record to direct OPAC users to reference 
1 iterature: 

1. Mischo's (1980) rotation algorithm applied .to the 
subject-rich fields of reference books' MARC records. 

2. Analysis of the 510 fields of CONSER records (which state 
what periodical indexes cover the journal) through the use 
of an algorithm similar to one used by the PaperChase OPAC 
to direct users to pertinent periodical indexes, especially 
in highly posted OPAC searches (see Vol. II, Figure 24). 

c. Focus group interview participants suggested the addition of 
campus or community information to the OPAC. 



G. User Assistance for Searching Online Public Access Catalogs 

1. Online and Offline User Assistance Provided by Participating Libraries 

t The types of online and offline assistance provided to OPAC users at 
the nine surveyed libraries and six libraries where focus group 
interviews were conducted demonstrated the variety and amount of 
assistance available. All libraries offered the assistance of 
reference 1 ibrarians who were stationed near the OPAC terminals. 
Only the Library of Congress offered a center for OPAC terminals 
where reference librarians' principal task was to answer inquiries 
about OPAC use. At other libraries, reference .1 ibrarians handled 
both reference and OPAC-use inquiries. Nearly all libraries featured 
printed materials on OPAC use, which were available for reference at 
the terminal or for the taking (see Vol. II, Table 57). 



2. Nonusers (and Why They Will Not Be Nonusers for Very Long) 

Over 75% of surveyed nonusers were likely to use the computer catalog 
in the future (see Vol. II, Table 59).. Library patrons who would not 
use the computer catalog in the future had selected "card catalog 
easier" or "dislike computers in general " as the reasons why they had 
not yet used the online public access catalog. In general, the main 
reasons why surveyed nonusers have not yet used the OPAC are that 
they have not taken training sessions on use, they have not had time 
to learn, or they had no need to use a library catalog (see Vol. II, 
Table 58). 

The majority of surveyed nonusers felt that it would take less than 
30 minutes to learn to use the OPAC. More nonusers at the five 
libraries where OCLC was the system under study than at libraries 
where local online access was studied (Syracuse, Ohio State, Dallas 
Public, Iowa City Public) thought that it would take more than 30 
minutes to learn how to use OCLC (see Vol. II, Table 60). The 
difference between the two groups might be attributed to the 
perceived complexity of the OCLC terminal with its many buttons, the 
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paucity of patrons actually seen using the OCLC terminal, or the 
location of terminals in reference departments. 

• A large percentage of nbnusers who felt that the OPAC would be 

difficult to learn also felt that it would take longer than one hour 
to learn (see Vol. II, Table 61). 

§ Many current nonusers of OPACs will eventually become users 

regardless of the changes' in libraries 1 methods' of assisting patrons. 
Libraries might have to change their methods of assisting patrons to 
win over nonusers who feel that the card catalog is easier to use 
than the OPAC and who dislike computers in general, such as providing 
increased amount of individualized instruction, e.g., introduction of 
a computer catalog center similar to the one at the Library of 
Congress, volunteer trainers, or fostering peer teaching for 
reluctant and sel f-conscious faculty. 



Online and Offline User Assistance 

• The majority of OPAC users at all surveyed libraries first heard 
about the OPAC by seeing a terminal in the library. A large number 
of OCLC users first heard about the OPAC from the library staff (see 
Vol. II, Table 62). 

• OCLC users and OPAC users at Dallas Public learned how to use the 
OPAC from library staff. In contrast, users at Syracuse and Ohio 
State relied primarily on printed material to learn to use the OPAC. 
Iowa City Public's OPAC had been cited by OPAC users for its online 
assistance in a discussion of OPAC features with which users had had 
favorable experiences (see Vol. II, Table 50). Users of Iowa City 
Public's OPAC learned how to use the system by consulting 
instructions on the terminal screen or in printed materials (see Vol. 
II, Table 63). 

• The source of assistance employed by OCLC users and Dallas Public 
users in their most recent search was the library staff. Syracuse 
and Ohio State OPAC users consulted printed material or signs. Iowa 
City Public's users either did not seek help, or they followed the 
instructions on the terminal screen to complete their most recent 
search of the OPAC. The experiences of Iowa City Public patrons 
contrasted with that of patrons at other libraries because the former 
relied on the available online assistance to perform their searches 
(see Vol. II, Table 64). 

§ Comparisons of the assistance used by very frequent users of the Iowa 
City Public OPAC with that used by very infrequent or first-time 
users showed similar patterns in the sources of assistance employed 
(see Vol. II, Tables 65 and 66). In contrast, many first-time users 
and infrequent users of the Syracuse, Ohio State, Dallas Public, and 
OCLC systems obtained the assistance of the library staff. The 
findings at Iowa City Public suggested that OPAC searchers would use 
online assistance when it was provided; this might also reduce 
searchers' dependence on the library staff, especially those 
searchers who rarely used the OPAC or used it for the first time. 
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4. Recommendations for Brochures, Information Sheets, and Other Printed 
Materials 

• Enhancements to OPACs regarding offline or online user assistance 
were gleaned primarily from focus group interviews. Surveyed users 
at Dallas Public and Ohio State called for printed materials to 
assist them at the computer catalog terminal (see Vol. II, Table 55). 
Since the latter already provided printed brochures for the taking, 
survey respondents probably wanted a command chart mounted on the 
terminal near the screen, similar to the charts on terminals at 
Mankato State. 

• An understanding of focus group interview participants 1 remarks about 
printed materials available in their libraries led to the following 
guidelines about the production of printed materials: 

1. Prepare printed materials of varying lengths. 

2. Include meanings for abbreviations used in OPAC records. 

3. Show examples of the major search approaches, i.e., author, 
title, author/title, subject, call number. 

4. Identify field labels and field information in examples of 
bibliographic record displays. 

5. Avoid small print, footnotes, jargon. 

6. Highlight the basics of subject searching. 

7. Enumerate the coverage and contents of the database and tell 
users when they should also consult the library's other 
catalog(s). 

8. Pretest printed material. 

5. Assistance from Library Staff 

• The experiences of library staff listed below describe the range of 
services they provided to. OPAC users who obtained their assistance: 

1. On-the-spot individual instruction. 

2. Answering reference inquiries. 

3. Getting the patron started. 

4. Troubleshooting. 

5. Group instruction. 
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6. Performing online searches for patrons, 

7. Individualized instruction by appointment, 

8. Preparation of offline user aids, 

t Library patrons expressed the value of consulting the library staff 
about OPAC-use problems. Experiences with volunteer trainers at 
terminals at Dallas Public, and Iowa City Public differed. Libraries 
considering volunteer trainers should provide workshops on at least a 
monthly basis ta help trainers exercise their OPAC searching skills, 
inform them of new developments in the library's services, and teach 
them new OPAC enhancements or search strategies. Also, volunteers 
could debrief library staff about patrons' problems, which could help 
in the enhancement of the system or the planning of new services. 

t Patrons at 1 ibraries where there were no formal OPAC workshops or 
classes expressed interest in attending formal training sessions. 
The major reason nonusers had not yet used the OPAC was that they had 
not yet attended training sessions on OPAC use. Thus, training 
sessions would probably be attended by these library patrons, 

6, Online User Assistance 

t The importance of online user assistance was emphasized in the 
comparison of user assistance sought by frequent users and by 
infrequent or first-time users (see Vol, II, Tables 65 and 66), The 
availability of online user assistance might reduce dependence on the 
library staff by first-time or very i nfrequent library users. 

• The following suggestions on how to improve online assistance 
capabilities of OPACs were elicited from focus group interview 
participants: 

1. Computer-assisted instruction programs. 

2. Dialog modes for patrons based on their experience or frequency 
of OPAC use. 

3. Informative error messages that explain the nature of the error 
and how to enter the i nformation , correctly . 

4. Prompts that direct the patron to the next logical step in the 
ongoing search and/or suggest possible actions to take. 

5. User-oriented interface that is like a helpful companion working 
together with patrons to find library materials rather than 
working against them. 

7. Relearnabil ity of the online public access catalog was a theme that 
emerged in focus group interviews with library patrons and staff and is 
supported by statistics on the frequency of library visits, OPAC use, and 
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use of computer systems other than the computer catalog. Online user 
assistance such as computer-assisted instruction, directive and suggestive 
prompts, and multiple-dialog modes might be the most successful means of 
making sure that online public access catalogs would be easy and painless 
for library patrons and staff to relearn. Research with the concept of 
supplantation and online retrieval systems is another promising approach 
to solving the problem of relearnabil i ty. Improving the relearnability of 
existing OPACs and ensuring it in future OPACs is a key issue in the 
development of online public access catalogs. 
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3.0 APPLICATION OF FINDINGS 

/ ' •: 

The extensive findings from the study are summarized below. Two methods of 
presentation have been selected: (1) brief, one-sentence statements 
generalize findings from the questionnaire, interviews, and transaction logs; 
and (2) specific findings are summarized in question and answer format. 



3.1 BRIEF GENERALIZATIONS FROM' FINDINGS 

These generalizations are offered as a much simplified view of the results of 
the study; as such, they do not reflect the variation found among the 29 
library systems surveyed. AHdTtional ly, the reader is advised to observe the 
following caveats: (1) although the project sample size is large, the return 
from some of the single library systems was small (less than 100); (2) the 
questionnaire has yet to be validated with repeated experimentation; and (3) 
the transaction log and focus group interview data, while generally validating 
the findings from the questionnaires, is not conclusive, as the data- comes 
from only 4 and 6 of the 29 study institutions, respectively. With these 
caveats, the following generalizations are offsred: 

1. In all types of libraries, patrons are conducting more subject searches 
than most librarians generally believe they do. (And more than most 
previous studies have shown.) 

.2. "What is in the database?" is a question for which patrons do not have a 
clear or correct answer. 

3. Patrons will use OPACs to browse tables of contents, book summaries, and 
books' indexes whenever we add them to the database. 

4. Patrons like to have the "how to use the system" information on the 
terminal or right next to it. 

5. The lost arts of lap-writing and writing while standing are making 
comebacks because the terminal work space is not always designed to 
provide a place to write. 

6. Patrons would like to have printouts of their searches - immediately (not 
the next day), so that they needn't write out search results. 

7. Many people do not use a catalog frequently enough to remember how to use 
it, so the system should have an interface specifically for the new and 
infrequent user as well as for the expert or frequent user. 

3, Locate the terminals where people can see them, since if they see them, 
they will use them. 
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9. Some people feel the lack of privacy when using the OPAC because anyone 
can see. what is retrieved and displayed on the screen. 

10. People do not generally consult full bibliographic record displays. 

11. Patrons do not like or understand many of the abbreviations used in 
bibliographicVexord and circulation record displays and with regard to 
command names. 

12. People browse the online catalog. 

13. Patrons believe the online catalog is fast and easy to learn to use even 
if they have not used it before. 

14. A great. majority of patrons learn to use the system on their own, some 
with a little help from a friend or the librarian. _ 

0 

15. Online catalogs are preferred over the card catalogs by OPAC users. 
(Nonusers are as yet undecided.) 

16. Patrons want more items in the database such as circulation status and 
magazine articles. 

17. More terminals are requested and some patrons want to access the OPAC from 
home. 

18. Patron expectations are high and will be even higher in the f ^ ure u wnen . . 
they see what can be done to enhance existing OPACs and when they have had 
the opportunity to use more than one OPAC or online retrieval system. 
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3.2 SPECIFIC FINDINGS IN QUESTION AND ANSWER FORMAT 

These questions were obtained from the libraries working with OCLC on this 
project. The questions have been categorized into logical sets for ease of 
reading. Where appropriate, the sections in the two major volumes of this 
report are noted so the reader may locate the complete discussion of the data 
collected. The answers provided here are therefore brief by design and are to 
be used with the supplemental information cited. 

CATEGORY A QUESTIONS RELATED TO THE EVALUATION AND/OR DESIGN OF OPACS 



Question 1, 



Answer 



Question 2. 



Answer 



Question 3, 



What are. the differences between catalog users at different 
types of libraries that might influence the criteria for design 
and evaluation of OPACs for each type of library? 

There were no characteristics that differentiated catalog users 
at one type of library from those at another. However, within 
each type of library, differences did exist between frequent and 
less frequent OPAC users. Statistics showed that "...OPAC use 
was not an everyday task of library patrons at any, of the five 
types of libraries surveyed" (Vol. II, section 7.5). Therefore, 
"rel earnabil i ty, 11 or the ease of relearning the use of the 
system, was shown to be a key element of design and evaluation 
in OPACs for all types of libraries. Additionally, use patterns 
were noted in the transaction Jog analysis; however, no 
connection between individual users and their pattern of use was 
possible, as user anonymity was consistently protected for the 
sake of privacy. 

Is it preferabl e*to build in a separate authority or 
cross-reference file in an online catalog, or to make 
cross-referencing or linking transparent to users? 

Patrons prefer the cross-reference structure to be visible, 
though not intrusive. The switch from the term entered to the 
controlled term should not be transparent, but prefaced with a 
brief explanation. For example, the screen could read "World 
War 2" = "World War, 1939-1945 in this catalog". Note Vol. II, 
section 6.3.2.3, Online Displays of Related Subject Headings, 
for more information. 

How is catalog use expected to change over time in an online 
catalog environment? For example, will major emphasis be on 
known-item searching, or on subject/key word access? If the 
latter, how best does one approach this form of access, given 
the restriction of a controlled subject vocabulary designed for 
a card catalog? Can we bypass conventional subject access, 
using subject heading words in a free text environment, or 
should we stick to a strict use of subject access in the 
traditional sense? 
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Answer 



Question 4, 



Answer 



Question 5, 



Answer 



Question 6. 



Answer 



All the data points to a high use of subject access. Term word 
searching or searching by subject headings alone will not be 
sufficient. Subject headings are needed for the collocative 
functions, and term searching is needed when the patron only has 
one term in mind. However, the uncontrolled and controlled 
terms should be linked. For more information, note Vol. II, 
section 6.3.2.5, Automatic Linking of Free Text Terms to the 
OPAC's Controlled Vocabulary. 

If our computer indexes are bound by data available to us, i.e., 
LC subject headings, should we contemplate loading. thesauri of ^ 
commonly used terms or term combinations? 

There is a real need for a link between the controlled 
vocabularies and common tejnms, although the work will be costly. 
(Linkage through classification numbers would also prove 
helpful.) Without appropriate links, patrons will likely try 
one or two combinations and, assuming the library has nothing to 
satisfy their needs, discontinue the search. Currently research 
in this area is being conducted by Robert Niehoff of Battel le 
Memorial Institute. 

Should the online catalog simply replace the card catalog or 
should the circulation function be a part of it? What are the 
cost to benefit considerations? 

Yes, the circulation function should be added. The patron will 
save time .and probably expects that the circulation information 
is there anyway. This project did not address the cost or 
cost/benefit questions. Vol. II, section 6.3.3, Call Number or 
Shelflist Searching, and Vol. II, Table 54 should be consulted 
for additional information. 

At what point, in an attempt to respond to the more 
sophisticated needs of some catalog users, do we so complicate 
the system that it is virtually unusable by the vast majority of 
users? Should there be two or more versions? If so, what about 
cost factors? 

Patrons want and will use two or more levels of a system. There 
are currently at least two' groups of users (first 
timers/infrequent users and heavy daily/expert. user^) . We may 
want to have three levels, one for the first timers/infrequent 
users, one for the once every two weeks or so user, and one for 
the daily/expert user. The move from one level to the next 
should not be noted by the patrons; they should just learn more 
each time they use the system. It is suggested that libraries 
offer a variety of online and offline user assistance to 
accommodate the variety of personal learning style's and 
preferences of OPAC users. For more information, see the 
following two sections in Vol. II: 7.4.2, Assistance from 
Library Staff, and 7.4.3, Online User Assistance. Cost factors 
were not addressed in this study, although it is obvious that a 
system designed to provide tailored service to three levels of 

. - 68 



ERIC 54 



Report Number: 0CLC/0PR/RR-83/4 
1983 March 31 



Question 7« 
Answer 



Question 8, 
Answer 



Question 9. 
Answer 

Question 10, 

c 

Answer 



Question 11, 
Answer 



users would be more costly to implement and operate than a 
system to accommodate only one level of user expertise. 

What are the essential characteristics of the successful online 
catalog? 

We do not know the answer to this question yet. We are tinly 
beginning to learn some of the OPAC characti sties found 
desirable by library users; we do not yet know which are 
essential 

Is there a common, essential set of characteristics of the 
libraries where online catalogs are successful? 



Some of the characteristics of the different libraries were 
noted, but not at the level needed to answer this question, 
term 'successful' is one that needs to be defined with, great 
care; it is so often a value judgment. 



The 



Which online catalog has the most successful prompts? 
the characteristics that make them successful? 



What are 



Question 12. 



The Iowa City Public Library patrons give high marks to the 
prompts in their system. The characteristics of successful 
prompts are outlined in Vol. II, section 6.2.3, Online User 
Assistance. 

What are the characteristics of a successful command package? 
(Cursor location, punctuation, spacing, etc.) 

This question cannot be answered fully as long as the term 
'successful' remains undefined. Some of the factors to consider 
in the design of output control feature are (1) Mnemonic command 
names or abbreviations; (2) Common names or abbreviations that 
are easily understandable in view of the operation they perform; 
(3) Common entry procedures that are easy to remember; (4) 
Command entry procedures that are consistent from one command to 
the next; (5) Suggestive or directive prompts that inform the 
user, guide the user, through possible actions, and explain how 
to carry out the actions. For more on this topic note Vol. II, 
section 6.3.5, Output Control Features. 

Does the online catalog display cause problems? Which of the 
online catalogs have the most successful ('preferred) displays? 

Yes, online catalog displays cause problems. It is not possible 
to say which OPAC is the most successful or preferred in this 
area. But thefe is a good deal to report on output controls. 
For information in this area, see Vol. II, section 6.2.2, Output 
Control and section 6.3.5, Output Control Features. 

Does the inclusion in the catalog of information intended only 
for library staff cause problems for library patrons? 
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Answer 



Question 13. 



Answer 



Question 14- 



Answer 



Yes. A graphic answer to this question can be seen in Vol. II, 
section 6.2.2, Output Control', where a record for Arthur Haley s 
Roots is given for four different systems. In addition, quotes 
from "the focus group interviews are also supplied in this 
section. 

Does online catalog research have findings similar to earlier 
card catalog research, for example, the proportion of subject to 
known item (author/title) searches? 

No. The findings from earlier card catalog studies are not 
similar to the findings from the online catalog study; there is 
a greater proportion of subject searching in online catalogs. 
For the percentages, see Vol. I of this report. 

What "human factors" in the OPAC environment (terminals, 
furniture, work space, location, access, setting) influenced 
OPAC use and patron satisfaction? 

Patrons are very much aware of the lack of good "human factors" 
design in the OPAC. For example, there is little space to write 
or store belongings while using the catalog. Many of the 
systems respond in only a few seconds, then the patron often 
needs 15 minutes to write out the information. The result is 
long queues. For more information read Vol . II, section 6.3.8, 
OPAC Workstation Design Considerations. 



CATEGORY B 
OPACS 



QUESTIONS RELATED TO USER EDUCATION/TRAINING AND PROMOTION OF 



Question 1. 



Answer 



Question 2. 
Answer 



What techniques were found to be most successful in promoting 
the OPAC in the libraries studied? Were they different for 
different categories/levels of users? 

This study did not track over time the promotion of the 
different systems to their different patron groups. However, 
user assistance was addressed and information on this topic is 
covered in Vol. II, section 7. 

What are the best training methods for users? 

f 

We did not single out any 'best' training method. "The analysis 
of focus group interviews and questionnaires provided no 
evidence that there were right or wrong approaches to user 
assistance or that one approach was overwhelmingly acceptable or 
sufficient for assisting most OPAC users." This statement comes 
from Vol. II, section 7.4 of this report, Recommendations for 
Online and Offline User Assistance." For additional information 
on this question, reefer to Vol. II, sections 7.4.1, Brochures, 
Information Sheets, and Other Printed Materials; 7.4.2, 
Assistance from Library Staff; and 7.4.3, Online User 
Assistance. 
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Question 3. Were planned programs of patron education judged to be effective 
in the libraries studied? 

Answer In libraries where these planned programs were given, the 

patrons liked them. In libraries where they were not provided, 
patrons asked for them. 

Question 4. What printed aids were found to be most helpful? Could printed 
aids be used to entice a nonuser to become an CPAC user? 

Answer The printed aids that were helpful were short and to the point. 

This topic is covered in Vol. II, section 7.4.1, Brochures, 
Information Sheets, and Other Pri nted Material s. As for 
enticing the nonuser, we must first look at why people are 
nonusers. The five top reasons for nonuse, according to the 
questionnaire data, are the following: 1) I have not yet taken 
training sessions on use; 2) I have not had time to use it; 3) I 
have not needed to use any library catalog; 4) I did not know 
there was a computer catalog; and 5) visitor or infrequent 
library user. For more on the topic of the nonuser, refer to 
Vol. II, section 7.2, Nonusers (and Why They Will Not Be 
Nonusers for Very Long). 

Question 5. What types of staff assistance were found to be most effective? 
One-on-one training? Glasses? Were different types of 
assistance perceived more helpful (or less intrusive) by 
different categories of patrons? 

Answer Since there are so many learning styles, it is essential to 

provide all types of help. This question is much like questions 
1 and 2 above. For complete findings on the topic, see Vol. II, 
section 7.4, Recommendations for Online and Offline User 
Assistance. ' 

Question 6. What are the demographics of users who are more receptive to the 
online catalog? Who are more resistant? 

Answer The most receptive are young students and researchers who 

require comprehensive information. The most resistant are older 
people who have personal 1 ibraries. For a discussion of the 
user demographics note Vol. II, sections 5.2, Favorable 
Attitudes Toward the OPAC and 5.4, Unfavorable Attitudes Toward 
the OPAC. 

.Question 7. What differences were discovered among the users of the OPAC by 
subject area or department (as judged by departmental library 
used)? 

Answer Being the first of its kind, the research design for this study 

required aggregation of data at a broader level. If the data 
collected were divided by departmental library (which it should 
not be), the sample size would not be sufficient to support the 
analysis. 
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CATEGORY C QUESTIONS RELATED TO PATRON USE AND ACCEPTANCE OF OPACS 



Question 1. 



Answer 



Question 2. 



Answer 



Question 3. 
Answer 



Question 4. 
Answer 

Question 5. 
Answer 



What are the characteristics of the successful versus the 
unsuccessful OPAC user? Of the user versus the nonuser? Of the 
satisfied versus the unsatisfied or dissatisfied OPAC user? How 
does the user judge the success of a search? 



All of this question can 
defined successful or un 
however, that patrons do 
of success of the search 
When a patron speaks of 
he or she refers to sati 
using a catalog correctl 
(demographics, attitudes 



not be answered since we have not 
successful searching. Ue did find, 
not always^eparate the success or lack 
from the nnding of needed information, 
success in the context of library use, 
sfying an informational need, not just 
y. See Vol. II for information 
etc.) regarding users and nonusers. 



Are there patterns in the success and/or failure of searches in 
the online catalog? What are the patterns? Do they differ 
among types of OPACs? How? 

With regard to search patterns, the transaction logs for a few 
systems show that patterns do exist. These patterns are noted 
for both the user who stays on the system for a time and for 
those who just seem to make errors. For more on this, note 
Vol. I. 

Can we meet the users' expectations for the new catalogs? 

Patrons expect a great deal, and although they are willing to 
give libraries some time, they may well find other information 
providers if we fail to meet their expectations. Vol. II, 
section 5.1, General Attitudes Toward the OPAC, provides a 
picture of the users' expectations vis-a-vis the library. 

What are the similarities and differences between the use of the 
online and card catalog? 

The data collected for this study has not been studied from this 
point of view yet. The questionnaires asked people to recall 
only their last search and did not seek comparisons. 



What is the user 
catalog? 



1 s next step when unsuccessful at the online 



This question was not formally addressed by this study. 
However, from the transaction log data analysis, we know that 
when a person makes an error using an OPAC, there is a 50% 
chance that his or her next action will also be. an error. To 
follow up on this question, refer to Vol. II, sections 5.0 and 
6.0. 



Question 6. Who does the patron blame for an online catalog failure? 
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Answer 



Question 7. 

Answer 
Question 8. 

Answer 
Question 9. 



Answer 



Question 10, 
Answer 



Question 11. 
Answer 



Patrons blame both the system and themselves for not getting the 
OPAC to work. Frustration can be read from the transaction 
logs, because there are sometimes four-letter messages to the 
library and the system when someone cannot get the OPAC to 
function. We also heard patrons in the focus group interviews 
say that they know the library had a given book but they could 
not get it to come up on the system. 

What are the significant differences among libraries in the 
acceptance of online catalogs both by library staff and by 
users? 

This topic was not studied. 

What library support is necessary to facilitate successful 
patron use of the online catalog? 

Vol. II, section 7.4, Recommendations for Online and Offline 
User Assistance, addresses this question. The key is a 
diversity of approaches. 

Were users generally found to tvave conceptualized their search 
well (effectively and efficiently) in system terms? Were any of 
the systems particularly helpful in assisting the user to 
formalize a search request? 

This research effort did not attempt to answer this question, 
but the Subject Access study just completed by Dr. Markey of 
OCLC did address this issue. Her report, "The Process of 
Subject Searching in the Library Catalog: Final Report of the 
Subject Access Research Project,' 1 is report number 
0CLC/0PR/RR-83/1 in the OCLC Office of Research Report series. 
This and other research reports are available from ERIC and 
OCLC. 

Did the users generally seem aware of system capabilities and 
coverage? 

Patrons have yet to learn to use all the OPAC capabilities. The 
command usage data collected in the transaction log analysis 
document this. For a full accounting of this, see Vol. I. 
Additionally, few patrons understand the content of the OPAC. 
Full information, on this topic is addressed in Vol. II, section 
6.3.7, Offline User Assistance. 

What elements of the record does the user feel are important? 

This topic was not studied in this research effort. Other 
studies report that patrons do not understand the different 
parts of a bibliographical record. There is a request from 
patrons for a short and simple record. 
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Question 12, 



Answer 



Question 13. 

Answer 
Question 14. 

Answer- 
Question 15. 
Answer 



Question 16. 
Answer 



Question 17. 
Answer 

Question 18. 



Do they understand the concept of subject headings and how they 
affect searching? Subject and name control? Thesauri? 

Some do and some do not understand subject heading, name 
control, and thesauri. The research suggests that we should not 
require this knowledge of patrons, but should build systems 
which supply it. For an understanding of the data collected in 
this area, refer to Vol. II, sections 6.3.1, Subject Access, and 
6.3.2.5, Automatic Linking of Free Text Terms to the OPAC s 
Controlled Vocabulary. 

Were users able to locate corporate authors or government 
publications more easily in the OPAC than in. the card catalog? 

This was not addressed in the study. 

Was online assistance available to the user "n any of the 
systems— and was it used/perceived to be useful? 

Yes, online assistance was available and was used by the 
patrons. For a full explanation, refer to Vol. II, sections 
6.2.3, Online User Assistance, and 7.3, Online and Offline User 
Assistance. 

At what point in a search would assistance be most useful? 

We do not have a complete answer to this question, but we do 
know that patrons want and need assistance when they have not 
found items they need or have located too many items. Vol. II, 
section 6.3.4, Reducing or Increasing Search Output, addresses 
the matter. 

What are user expectations for response time? Is response time 
ever too fast? 



This topic was not addressed s 
other research efforts report 
much more than 8 to 12 seconds 
often, response time is seldom 
interviews in this research, i 
be too fast if the patron does 
changed. This occurs when the 
the same or almost the same as 
does not notice the screen fli 



pecifically by this study, but 
that people do not want to wait 
For those who use systems 
too fast. During the focus group 
t was note(J that response time can 
not know that the screen has 
response to the current query is 
the last query and the patron 
cker. 



How can downtime be made minimally inconvenient to 'the user? 

This question was not specifically addressed, although comments 
suggested that preserving search results on the screen- when the 
system goes down would be particularly helpful . 



Can the user distinguish downtime frqm slow response? 
what can be done about it? 



If not, 
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Answer This was not studied. However, we have noted people 'terminal 

hopping 1 when a system goes down; that is, moving from one 
terminal to another looking for one that works. It may be best 
to post in some central spot that the system is down. 

Question 19. Can the impact of the OPAC real ly be judged when an alternative 
catalog exists? 

Answer This was not studied but people have been willing to wait in 

line for a terminal even if the card catalog is in the next 
room. When users were asked to compare the OPAC to other 
catalogs, most said the OPAC was better. For a complete 
explanation, see Vol. II, Table 43 (CLR- and OCLG-Aqgregates: 
Comparison of Library Catalogs Categorized by Users Academic 
Affiliation) and Table 44 (CUfc-Aggregate and Individual 
Libraries: Comparison of Computer and Traditional Library 
Catalogs) . 

CATEGORY D QUESTIONS RELATED TO THE IMPACT OF THE OPAC ON OTHER LIBRARY 
OPERATIONS AND SERVICES 



Question 1. What is the impact of the online catalog on other aspects of 
library service? For example, on the demand for reshelving, 
searches, holds, reorders, etc. 

Answer There is and there will continue to be an impact on other 

library services. For example, if circulation data is reported 
on the OPAC, patrons will expect the book to be on the shelf if 
they note that it is available. We will need to shelve returned 
books very soon after their status has been changed from 
circulating to available. Vol. II, sections 6.3.7, Offline User 
Assistance, 6.3.8, OPAC Workstation Design Considerations, and 
6.4, Enriching the OPAC's Database provide useful information on 
this topic. 

CATEGORY E QUESTIONS RELATED TO THE LARGER ISSUES: HOW WILL THE OPAC AFFECT 
THE FUTURE OF LIBRARIES AS INSTITUTIONS/ORGANIZATIONS 



Question 1. What are the implications for major changes in library service 
and perceptions of what the library is, as its catalog becomes 
widely available through terminals outside the physical library? 



Answer 



An essay could be written on this topic, but in general, people 
will expect more from their libraries. They now want more and 

The personal computer revolution will 
on the OPAC, since in some cases patrons 
their homes and offices. One need only 
spend a few minutes with these reports to see that OPACs are 
changing our libraries. 



better subject access 
soon have its effects 
now use the OPAC from 
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CATEGORY F QUESTIONS RELATING TO COST AND FINANCING OF OPACS 



How will we control the costs for the use of the system and who 
should pay? (It is generally believed the OPAC use should be 
free to patrons, though the cost of that service is generally 
higher than that of maintaining the card catalog and related 
services. Should patrons be asked to "cost-share" for the new 
service? ) 

This question was not addressed by this study. It is a major 
issue with current economic conditions and budget cutting 
philosophies and it will have to be addressed soon. 

How many terminals will we need to have? What operations can be 
cut back in other areas of the library and the attendant 
cost-savings used to finance the OPAC? 

It is possible to determine the required number of terminals if 
the arrival times, service volume* and pattern of use 
(distribution) are known. Such patterns have been determined in 
previous studies (e.g., Terminal Requirements for Online 
Catalogs in Libraries, an OCLC study sponsored by the National 
Science Foundation). The average service times for five OSU 
libraries ranged from just under 4 to just over 9 minutes. For 
Syracuse, the median service time was 3.55 minutes. For Dallas, 
the median was near 3 minutes. Vol. I, Chapter 6 (Conclusions) 
addresses the amount of time spent at the terminal by patrons. 
Refer in particular to Questions 1 and 2 in Chapter 6. 

Can the data from this study be generalized to the population as 
a whole? 

No. During this study we talked with, provided questionnaires 
to, and recorded the online catalog use of library patrons only. 
Generalizing to the population would have required that we 
sample from the entire population, i.e., nonlibrary, as well as, 
library users. 



Question 1. 

Answer 
Question 2. 
Answer 



Question 3. 
Answer 
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4.0 AREAS OF ADDITIONAL RESEARCH 



This study marks only the bpginning of the research that must be done to 
provide library patrons wi* the best online catalogs possible. The tools we 
have used, survey questionnaires, focus group interviews and transaction log 
analysis, must be used again and again over the next few years at the same and 
at different libraries in order to amass the knowledge needed to provide an 
acceptable human-computer interface. The need for this additional research is 
clear from the number of unanswered questions noted in section 3 of this 
volume. It is vital that we continue this research, share the results, and 
study what others in fields related to human-computer interaction are 
learning. 

In particular, future research should follow the attitudinal changes of 
patrons about OPACs. We have learned that the attitudes of patrons who use 
computers other than the library's computer catalog are not much different 
from those of patrons who do not use computers other than the library's OPAC. 
We expect changes in attitude between these two groups as personal computers 
grow in popularity and everyday use. Changes in patrons 1 attitudes may also 
occur when they have had an opportunity to use more than one OPAC. Today, 
with as many as four different OPACs available in the libraries of one city, 
such a comparative study soon will be possible. 

Another rich area for research is the study of transaction logs over time to 
see the effects of changes in a given system. For example, we have learned 
that some online error messages and online help screens are not particularly 
helpful and that patrons who make one error are very likely to make another 
error. When changes are made in OPACs in which transaction log data is 
available, we will be able to study the effectiveness of these system changes. 
The opportunity to monitor our modifications in an unobtrusive manner will 
make it possible to improve patrons 1 ability to access information. 
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APPENDIX A. FINAL USER QUESTIONNAIRE 



Council on Library Resources 
COMPUTER CATALOG STUDY 
User Questionnaire 



The library is conducting a study of its computer catalog to improve it. This question- 
naire is a way to communicate your views. It should take you only about 15 minutes 
to complete. Your responses are confidential. Please do not write your name anywhere 
on the questionnaire. Thank you. 



MARKING INSTRUCTIONS 










• USE A NO. 2 PENCIL ONLY. 








• FILL THE CIRCLES COMPLETELY. 








• BE SURE TO ERASE CLEANLY ANY 




MARKS YOU WISH'TO CHANGE. 








• MAKE NO STRAY MARKS ON THIS 




QUESTIONNAIRE. 






0 




o 




o 




o 



■oioowooiionoooooooooo 13925 

00 NOT WRITE IN THIS AREA 



NCS Trant Optic 05 15301 321 
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PART 1: ABOUT YOUR MOST RECENT SEARCH 
INSTRUCTIONS: Plesse snswer these questions sbout the computer catalog search you just 
completed. 



1. I came to this computer search with: 
(Mark ALL that apply) 

■ A>complete author's name O 

b Part of an author's name o 

. c A complete title o 

d Part of a title O 

e A topic word or words o 

f A sub|ect heading or headings .0 

g A complete call number o 

h Part of a call number o 



2. By searching this computer catalog I was trying 
to find: 

(Mark ALL that apply) 



a A specific book journal or magatme 



o 



b Books journals or magazines on a topic or subject O 
c Books Dy a specific author ... o 



d Information such as publisher date, spelling 
of a name etc 

e II a book that I know the library has is available 

for my use 

f Another library that has a book journal or 
magazine mat i want 



3 I searched for what ( wanted by: 
{Mark ALL that apply) 



a A complete author's name 
b Pari of an author s name 

c A complete title 

d Part of a title 

e A topic word or words 

f A subject heading or headings 

g A complete call numb*; 

h Part of a call number 



o 
o 
o 



o 

:..o 
..o 
o 

'.O 

...o 
o 
o 



4. I need this information for: 
(Mark ALL that apply) 

a Recreational uses O 

b Making or fimng something n O 

c My work or job o 

d Personal interest O 

e A hobby . ...O 

f Class or course reading O 

g A course paper or report O 

h A thesis or dissertation o 

i Writing for publication . . . . o 

j Teaching or planning a course o 

k Keeping up on a topic or subject o 



5. In this computer search I found: 
(Mark ONE only) 

a More than I was looking for 

b All that I was looking for 

c Some of what I was looking for . . . 
d Nothing I was looking for 



o 
o 
o 
o 



6. In relation to what 
search was: 
(Mark ONE only) 



a Very satisfactory 

b Somewhat satisfactory 

c Somewhat unsatisfactory 

d Very unsatisfactory 



was looking for. this computer 



o 
o 
o 
o 



7 I came across things of interest other than what I was 
looking for: 



a YES 
b NO 



O 
O 



8 I got help in doing this computer catalog search from 
(Mark ALL that apply) 



a Printed material or signs . 

b Instructions on the terminal screen 5 . 

c Library staff memoer 

d Person nearby ... 

e I did not get help 



o 
o 
o 
o 
o 
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9. My ovefell or cjanaraf attreuda toward tha computar 
catalog is: , 
(Mark ONE only) 

a Very favorable 

b Somewhat favorabJa 

c. Somewhat unfevoreble 

d Vary unfevoreble 



10. Compared to tha card, book, or microficha catalog 
in this library, tha computar catalog it: 
(Mark ONE only) 



.0 
.0 

.o 
.o 



a. Banar 

b. About tha aama 

C. Worst 

d. Can't dacida ... 



.0 
.0 

.o 

.0 



PART 2: YOUR EXPERIENCE WITH COMPUTER CATALOG FEATURES 
INSTRUCTIONS: Mark tha aingla column for aach quattion that corraaponda moat ctoaaly to how 
you faal. If tha statamant doaa not apply to your axparianca at tha computar catalog, mark tha 
column. "Doaa Not Apply". 



traoNoiv 

AORCI 



NCrrxiR 
MM I NO* 
04 14 G Ml I 



IT 

o 
o 
o 
o 



o 
o 
o 
o 
o 



o 
o 
o 
o 
o 



ST MO HOLY 
OOAORIE 



XT 

o 
o 
o 
o 



o 
o 
o 
o 
o 



o 
o 
o 
o 
o 



ooct 

HOT 



1 A computar search by title is difficult 

2 A computer starch by author is aasy 

3 A computar saarch by subject it diHicult 

4 A computer search by cell number is easy 

5 A computer saarch by combined author /title is difficult 



16 



Remembering commends in tha middle of the saarch is aasy . . 

17 Finding the correct subject term is difficult 

18 Scanning through a long display (forward of backward* is eesy 

19 Increasing the result when too little is retrieved is difficult 

20 Reducing the result when too much is retrieved is eesy 



Understanding explanations on the screen is difficult 

Using codes or abbreviations for searching is eesy 

Abbrevietions on the screen ere essy to understend 

Loceting call numbers on the screen is difficult 

Seerchmg with a short form of e neme or e word (truncation) is easy 



o 
o 
o 
o 



o 
o 
o 
o 
o 



o 
o 
o 
o 
o 



o 
o 
o 
o 



o 
o 
o 
o 
o 



o 
o 
o 
o 
o 



o 
o 
o 
o 



o 

.0 

o 
o 
o 



o 
o 
o 
o 
o 



o 
o 
o 
o 



o 
o 
o 
o 
d 



o 
o 
o 
o 
o 



STNONOIV 



NirTMiR 

AONIE HO* 
OltAQMt 



3 TWO HOLY 

oiSAontt 



OOES 
WOT 



26 Using logical terms like AND. OR. NOT is difficult 

27 Remembering the e*ect sequence or order of commends is eesy 

28 Understending the initial instructions on the screen is difficult 

29 Understending the display for e single book, journel or megenne is eesy 

30 Understending the display that snows more than e single book, 
journel or magazine is difficult 

31 Interrupting or slopping the d*spley of information is eesy 

32 Typing in exact spelling, initials, spaces end hyphens ts difficult to do . . 

33 Knowing whet is included in tha computer catalog is eesy to remember 

34 The order in which items era displeyed is easy to understand 

35 Displeyed messages era too long 



IT 
o 
o 
o 



o 
o 
o 
o 
o 



IT 

o 
o 
o 



o 
o 
o 
o 
o 



o 
o 
o 



o 
o 
o 
o 
o 



o 
o 
o 
o 



o 
o 
o 
o 
o 



o 
o 
o 



o 
o 
o 
o 
o 



o 
o 
o 
o 



o 
o 
o 
o 
o 
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36 Selecting from a list of choices takes too much time 

37 Entering commands whan I wanl lo during tne search process is 
difficult 

38 The rate at which the computer responds is too slow 

39 The availability of signs and brochures is adequate 

40 Signs and brochures are not .very useful 

41 The staff advice is often not helpful. 

42 It is hard to find a free terminal 



STRONGIV 
AONCC 


AO*U 


mithim 

AQNCI NO* 
011*0*11 


OltAOMI 


STRONGLY 
OISAGMC 


oocs 

NOT 


. 0 


0 


0 




6 


0 


o 


o 


O 


o 


o 


o 


o 


o 


O 


o 


o 


o 


o 


o 


O 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 



YOU ARE MORE THAN HALF - WAY DONE 




PART 3: IMPROVING THE COMPUTER CATALOG 
INSTRUCTIONS: Selact the response or responses thai best reflect your views about changes" 
that should be made in the computer catalog. 



i 43. Whan I use the computer catalog terminal 
, ' (Mark Y_£S or HQ) 



a The keyboard is confusing to use 

b There is too much glare on the screen . . ■ 

c The letters and numbers are easy to read 

d The lighting around the terminal is too 
bright 

e There is enough writing space at the 
tarmtnat 



f Nearby noise is distracting 

g The terminal table is too high or too low 
h The printer is easy to use 



YES 


NO 


0 


0 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 



45. Select up to FOUR computer catalog service improve- 
ments you would like the library to make; 



■44 Select up to FOUR additional features vou 

■ like this computer catalog to have: 

■ a Providing step by 4tep instructions 

1 b Searching by any word or words in a title 

c Searching by any word or words in a subject 

■ heading ... 6 
■ 

■ d Limiting search results bv date of publication 
1 e Limiting search results by language 

■ f Ability to search by (ournal title abbreviations 

" g Ability lo change the order in which items are 

■ displayed 

h Ability to view a list of words related to my 

■ search words . , 

" i Ability to search for illustrations and 

■ bibliographies 

■ } Ability to search by call number 

■ k Ability to print Search results 

" I Ability to search a boo* s table of contents 

■ Summary or index .... 

■ m Ab'lity to know if a boo* is checked out 

" n Abilu, w tell where a book is tocated in the 

■ library 

■ o None 



would 



o 
o 



o 
o 

o 

o 

o 

o 
o 
o 

o 
o 

o 
o 



a More terminals - 

b Terminals at locations other than near the card catalog 

c Terminals at places other than library buildings 

d A charrof commands posted at the terminal 

e A manual or brochure at the terminal 

f An instruction manual for purchase ... 

g Training session'* ... 

h Shde tape cassette training program 

i None 



o 
o 
o 
o 
o 
o 
o 
o 
o 



46 Select up, to FOUR kinds of material you would like to 
see added to the computer catalog: 



A7. BRIEFLY DESCRIBE ANY OTHER PROBLEMS WITH 
THIS COMPUTER CATALOG OR CHANGES YOU 
WOULD LIKE MADE TO IT: O 



a Dissertations 

b- Mohon picture films 

c Government publications 

d Journal or magazine titles 

e Maps ...... 

I Manuscripts 
g Music scores 
h Newspapers 

> Phonograph records o r tapes 

I Technical re»"-<«ts 

k More of the library s older books 
t None 
m Other 



o 
o 
o 
o 
o 
o 
o 
o 
o 
o 
o 
o 
o 
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PART 4: ABOUT YOURSELF 
INSTRUCTIONS; Your responses ire confidential. Please do not write your name anywhere on 
this questionnaire. 



48. I comi to this library: 

a Daily / O 

b Wtthly 

c Monthly ^ 

d Aboul lour times a year ^ 

e About once a year O 

» Not before today • ^ 



49. I use this computer catalog: 

a Every library viSH ^ 

b Almost every visit ^ 

c Occasionally ~~ 

d Rarely £j 

e Not before today 



SO. I use this library's book, card or microfilm catalog; 



a Every visit 

d Almost evcf V visit 

c OccasionaMy 

d barely 

e Never 



o 
o 
o 
o 
o 



51. I use a computer system other than the library's 
computer catalog. 



d Daily 
b Weey'v 
C Montnly 

d About lour hmes a year 
e About once a year 
i Never 



52 i first heard about this computer catalog from 
(Mark ONE only) 



a Noticing a terminal in the library 

b Library tour orientation or demonstration 

c An article or written announcement 

d A course instructor 

e A friend or lamily member 

I LtUrary stall 



o 
o 
o 
o 
o 
o 



o 
o 
o 
o 
o 
o 



53. I learned how to use this computer cetelog: 
(Merk Mk <h »' »PP'V) 

a From a Iriend or someone at a nearby terminal O 

b Using printed instructions o 

c Using instructions on the terminal screen 

d From the library staff 

e From a library course or orientation • 

I From a slide tape/cassette program 

g By mysell without any help 



54. My age group is: 



o 
.o 
o 
.o 
o 



d 14 and under 
b 15* 19 years 
c 20 24 years 
34 vears 
44 years 
54 years 
64 years 



(1 25 
* 35 
f 45 
g 55 



h 65 and over 



.o 
o 
o 
o 
o 
o 
.o 
.o 



55 I am 



a Female 
b Male 



.o 
.o 



56 Mark your current or highest educational level: 
{Mark ONE only) 



a Grade School or Elementary School 
b High School or Secondary School 

c Some College or University 

d College or university Graduate 



o 
o 
o 
o 



If you are not completing this questionnaire at a college or 
university, please stop here. Thank you. 



If you are completing this questionnaire at a college or uni- 
versity, please continue. 



57. The category that best describes my academic area is 
(Mark ONE only) 



a Arts and Humanities 

b Physical- 'Biolog cal Sciences 

c Social Sciences . 

d Business Management ... 

c Education ... 

I Engineering ... 

g Medical Health Sciences . 
h Law , 
i Maior not declared 
j Interdisciplinary 



o 




o 




o 




.o 




o 




■ C 




.o 




.o 




.o 




o 
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& 

58 Tha main focua of my acadamic work at tha prasant 

tirha is: 

(Mark ALL that apply) 

a. Coursa Work O 

b Teaching O 

c Rasaarch O 



59. My praaant affiliation with thia collaga or univaraity 
ia: 



a Fraihman/Sophomora O 

b. Jumor/Samor O 

c Graduata • mat tars (aval O 

d Graduata • doctoral laval O 

a Graduata • profattional school O 

f Faculty O 

9 Staff v . O 

h Othar O 



Thank you for participating in this study of the computer catalog. This completes the 
questionnaire. Please return it. 



SUPPLEMENTARY QUESTIONNAIRE ITEMS 

so ooooooooooooooo 

61 ooooooooooooooo 

62 ooooooooooooooo 

63 ooooooooooooooo 

64 ooooooooooooooo 
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APPENDIX B. FINAL NONUSER QUESTIONNAIRE 



Council on Library Resources 
COMPUTER CATALOG STUDY 
Questionnaire 



The library has a computer catalog and not everyone has had a chance to use it yet. If you have 
NEVER USED THE COMPUTER CATALOG you can contribute to the quality of library services 
by completing this questionnaire. It takes about five minutes. Your responses are confidential. 
Please do not write your name anywhere on the questionnaire. Thank you. 



MARKING INSTRUCTIONS 








• USE A NO. 2 PENCIL ONLY 




• FILL THE CIRCLES COMPLETELY. 


/ 






• BESURETO ERASE CLEANLY ANY 




MARKS YOU WISH TO CHANGE. 


r/ 






• MAKE NO STRAY MARKS ON THIS 




QUESTIONNAIRE 






o 




o 




o 




o 



0M1OB1MOOBOBOOOOOOOOOO 

DO NOT WRITE IN THIS AREA 
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PART 1: WHAT YOU THINK ABOUT 

THE COMPUTER CATALOG 
INSTRUCTIONS: Please mark the response 
that best describes how you view a computer 
catalog. 



1. I have not used the computer catalog up to now 
•because: 

(Mark ALL that apply) 



a I do not like to use computers 

b I did not know there was a computer cMalog 



c I do not know where it is 

d I have not had time to learn to use n 



.o 
.o 

o 

.0 



o 
o 



e I have not taken training sessions on how to use it O 

f There has not been any staff at the terminals to 
assist me in using it 

9 The terminals were all in use when I wanted to 

use it 

h I have not needed to use any library catalog 

recently 

i The card catalog is easier to use 

I The card catalog contains more of the information 
I need 

k I am a visitor or infrequent user of this library .... 



o 
o 

o 
o 



2. How much time do you think it takes to learn to use 
the computer catalog? 



a A day or more 

b Between I 2 of a day and a day 
c Between an hour and 1 2 of a day 
d Between 30 minutes and an hour 
e. Between 15 minutes and 30 minutes 
f 1 5 minutes or less 



o 
o 
o 
o 
o 
o 



3 How difficult or easy do you think it would be to learn 
to use the computer catalog? 



a very difficult 
b Somewhat difficult 
c Somewhat easy 
d very easy 



o 
o 
o 

o 



4. My overall or general attitude toward the computer 
catalog is: 

a Very favorable o 

b Somewhat favorable o 

c Somewhat unfavorable o 

d Very unfavorable o 



5. How likely are you to use the computer catalog in the 
future? 

a very fikely , . O 

b Somewhat likely o 

c Somewhat unlikely o 

d Very unlikely o 



6. Compared to the card., book, or microfiche catalog 
in this library the computer catalog is. 
(Mark ONE only) 

a Belter O 

b About the same o 

c Worse O 

d Can't decide ■ . o 



PART 2: ABOUT YOURSELF 
INSTRUCTIONS: Your responses are con- 
fidential. Please . do not write your name 
anywhere on this questionnaire. 



7. I come to this library: 

a Daily J . . O 

b Weekly O 

c Monthly O 

d About four times a year ..- o 

e About once a year o 

f Not before today O 



8 I use this library's book, card or microfilm catalog: 



a Every visit ... 

b Almost every visit 

c Occasionally 

d Rareiv 

e Not before today 



o 
o 
o 
o 
o 
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9. I use a computer system othor than the library's 
computer catalog 

a Daily ' O 

b Waakly O 

c Monthly • : O 

d About four times a year O 

a About once a year O 

f Navar .* .' O 

10. My age group is: 

a 14 and under ^ 

b 15 19 years ^ 

c 20 24 years O 

d 25-34 years O 

e 35 . 44 years ' ■ ■ O 

f 45 54 years O 

g 55 64 years ^ 

h 65 and over O 

1 1 I am 

a Female O 

b Mate O 



12 Mark your current or highest educational level: 
(Mark ONE only) 

a Grade School or Elementary School . O 

b H«gh School or Secondary School O 

c Some College or University . O 

d College or University Graduate O 



If you are not completing this questionnaire at a college 
or university, please stop here. Thank you. 



If you are completing this questionnaire at a college or 
university, Please continue. 



13. The category that best describes my academic 
area is. (Mark ONE only) 

a Arts and Humanities o 

b Physical/ Biological Sciences ..o 

c Social Sciences o 

d Business/Management o 

e Education O 

f Engineering . o 

g Medical' Haalth Sciences O 

h Law O 

i Major not declared - o 

j Interdisciplinary Q 



14. Tha main focus of my academic work at the present 
time is: 

(Mark ALL that apply) 

a Course Work O 

b Teaching O 

c Research O 



15. My present affiliation with this college or university is: 



Freshman /Sophomore O 

Junior/Senior O 

Graduate • mastars level o 

Graduate • doctoral level o 

Graduate • professional school 

o 

Faculty O 

Staff ■ O 

Other .• O 



Thank you for participating in this study of the 
computer catalog. This completes the question- 
naire. Please return it. 



SUPPLEMENTARY QUESTIONNAIRE ITEMS 



16. QOOOOOOOOOOOOOO 

17. ooooooooooooooo 
is ooooooooooooooo 
19. ooooooooooooooo 
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COUNCIL ON Z 
LIBRARY RESOURCES 

COMPUTER i 

CATALOG | 

STUDY I 

USER QUESTIONNAIRE - 



THE LIBRARY IS CONDUCTING A STUDY 
OF ITS COMPUTER CATALOG TO IMPROVE 
IT. THIS QUESTIONNAIRE IS A WAY TO 
COMMUNICATE YOUR VIEWS. IT SHOULD 
TAKE YOU ONLY ABOUT 10 MINUTES TO 
COMPLETE. YOUR RESPONSES ARE CON- 
FIDENTIAL. PLEASE DO NOT WRITE YOUR 
NAME ANYWHERE ON THE QUESTION- 
NAIRE. THANK YOU. 



BYu 



DATE:- 



MARK1NG INSTRUCTIONS 

• USE A NO. 2 PENCIL ONLY 

• FILL THE CIRCLES COMPLETELY /£j 

• BE SURE TO ERASE 
CLEANLY ANY MARKS 
YOU WISH TO CHANGE 

• MAKE NO STRAY 
MARKS ON THIS 
QUESTIONNAIRE 
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PART 1 : ABOUT YOUR MOST RECENT SEARCH 

INSTRUCTIONS: Pleaaa answer th—m qu Ml ion a with respect to 
the LAST TIME you used tha com put ar catalog. 



t CAME TO THE SEARCH WITH: 
(Marie all that apply) 

a. A complete author's name O 

b. Pan of an author's nama ...O 

c. A complata trtia , O 

d. Pan of a title O 

e A topic wordfs) O 

f A subiact heeding(s) C 

g A complata calf numbar v ~ 

h Pan of a call numbar O 

i Other;-^ C 



'2. BY SEARCHING THE COMPUTER CATALOG I 

1 WAS TRYING TO FIND: 

1 (Marie all that apply) 

1 a. A spacific book, journal or magazine '„ 

1 b- Books, journals or magazines on a topic ' 

1 c Books by a spacific author 

1 d. Information such as publisher, data, spelling of a . 
1 name, etc . 

» e. If a book that I know tha library has is on tha shelfC 

1 f Another library that has a book, journal or 

1 magazine that I want * 

1 g What it is like to use the computer catalog v 

1 h. Other — 



I SEARCHED FOR WHAT I WANTED BY; 
(Mark all that apply) 

a. A complete author's nama 

b. Pan of an author's nama 

c. A complete title 

d. Pan of a title 

e. A topic word(s) 

f A subject headings) 

g A complete call number 

h Pan of a call number 

i Others^ ; 



4. I NEED THIS INFORMATION FOR: 
(Merit all that apply) 

a. Recreational uses 

b. Making or fixing something 

c. My work or job 

d. Personal interest 

e. A hobby 

f. Class or course reading 

g. A course paper or report 

h. A thesis or dissertation 

i. Writing a paper, article or book 

j Teaching or planning a course 

k. Keeping up on a topic or subject 

5. IN THIS SEARCH I FOUND: 
(Mark one only) 

a. More than t was looking for 

b. All or most of what I was looking for 

c. Some of what I was looking for 

d. Nothing I was looking for 

6. IN RELATION TO WHAT J WAS LCOKING FOR. 
THE SEARCH WAS: 

(Mark ona only) 

a. Very satisfactory 

b. Somewhat satisfactory 

c. Somewhat unsatisfactory 

d. Vary unsetisf sctory 

7. I CAME ACROSS THINGS OF INTEREST OTHER 
THAN WHAT I WAS LOOKING FOR: 

, ■' Yes 

b. No 0 

8. IN ADDITION TO THIS COMPUTER CATALOG 
SEARCH I HAVE ALSO USED OR WILL USE: 
i(Merk all that apply) 

a. A card, book or microfilm catalog 

b. Printed indexes 

c. None of the above 

d. Other:-, )■ 



I GOT HELP IN DOING THIS COMPUTER 
CATALOG SEARCH FROM: 
(Mark all that apply) 

a. i did. not get help 

b. Printed material or signs 

c. Instructions on the terminal screen • • • 

d. Library staff member 

e. Person nearby 
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10. MY OVERALL OR GENERAL ATTITUDE TOWARO 
THE COMPUTER CATALOG IS: 
(Mark on* only) 

a. Very favorable • 

b. Somawhat favorabia ; 

c. Somawhat unfavorabla 

d. Vary unfavorabla 



1 1 . HOW LIKELY ARE YOU TO USE THE COMPUTER 
CATALOG IN THE FUTURE? 

a. Vary likely 

b. Somawhat likaly 

c. Somawhat unlikaly 

d. Vary unlikaly 



PART 2; YOUR EXPERIENCE WITH COMPUTER CATALOG FEATURES 

INSTRUCTIONS: If you atrongly agree with a ttatamant. mark tha "STRONGLY AGREE" column. If you ttrongly 
disagree, mark tha "STRONGLY DISAGREE" column. If you agree but not strongly, or disagree but not ttrongly, 
mark either tha "AGREE" or tha "DISAGREE" column dapanding on which corraaponda to your opinion. If you 
naithar agraa at all nor dlsegre* at all. mark tha center "NEITHER AGREE NOR DISAGREE" column. Ramambar 
to mark tha tingla column for aach quaation that corresponds most closaiy to how you faal. If tha sutamant doas 
not apply to your axparianca at tha computar catalog, mark tha column. "DOES NOT APPLY". 



DOES NOT APPLY 



STRONGLY DISAGREE 



DISAGREE 



NEITHER AGREE NOR DISAGREE 



AGREE 



STRONGLY AGREE 



REMEMBERING THE CORRECT WAY TO ENTER: 

1 2. A mis saarch is difficult 

13. An author saarch is aasy 

14. A subisct saarch is difficult 

,1*5. A call numbar saarch is aasy 

16. A combined author/title saarch is difficult 

DURING A SEARCH: 

17. Ramambaring QDmmands in the middle of the search is easy ^ 

18 Finding tha correct subject term is difficult 

1 9. Scanning through a long display (forward or backward) is easy 

20. Increasing the result when too linle is retrieved is difficult . . . , 

21 Reducing tha result when too much is retrieved is eesy 

22, Understanding explanations on tha screen is difficult 

23. Using codes or abbrevietions for searching is easy 

24 Understending codes or abbreviations on the screen is difficult 

, 25, Library names or sbbreviations on tha screen are easy to understsnd 

26. Locating call numbers on tha screen is difficult 

27. Understanding explanations for searching with a short form of h name or a word (truncation) 
is eesy : • 

28. Understanding explanations for using logics* terms like AND. OR. NOT is difficult 

21. Remembering the exact sequence or order of commands is easy 

30. Understending the initial instructions on the screen is difficult 

31 Understanding the display for a single book, journal or magazine is easy 

32. Understanding the display that shows more than a single book, journal or 
magazine is difficult 

33. Interrupting or stopping the display of information is assy 

IN GENERAL: 

34 Typing in exect spelling, initials, spaces and hyphens is difficult to do 

35. Knowing what is included in the computer catalog is easy to remember 
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DOES NOT APPLY 



STRONGLY DISAGREE 



DISAGREE 



NEITHER AGREE NOR DISAGREE 



AGREE 



— IN GENERAL: 



STRONGLY AGREE 



36. The yMrs of publication that are included in tha computer catalog ara 

difficult to ramambar 

37 Tha ordar in which itama ara displayad is aasy to understand 

38. Displayad messages are too long 

39. Selecting from a. list of choices takes too much time 

40. Entering commands when I want to during the search process is difficult 

41 . Good typing skills are required to use the computer catalog 

42. Comments: 





AO 




AO 


•A 


AO 


♦A. 


AO 


*A 


AO 


I* 


AO 



PART 3 : YOUR EXPERIENCE AT THE COMPUTER CATALOG TERMINAL 
INSTRUCTIONS: Continue to mark your degree of agreement or disagreement ea In the previous section. 



DOES NOT APPLY 



STRONGLY DISAGREE 



DISAGREE 



NEITHER AGREE NOR DISAGREE 



AGREE 



STRONGLY AGREE 



43. It is hard to find a free terminal 

AT THE TERMINAL I USED; 

44 The keyboard is confusing to use 

45. There is too much glare on the screen 

46. The letters and numbers are'easy to read 

47 The lighting around the terminal is too bright 

48. The writing space at the terminal is adequate 

49 Nearby noise is distracting 

50. The height of the terminal table is about right 

51. The rate at which the computer responds is too slow 

52. The availability of signs and brochures is adequate . 

53. Signs and brochures are not very useful 

54. Library staff are usually available to assist me 

55. The staff advice is often not helpful 



AO H 
AO H 
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IF THERE IS NO PRINTER AT YOUR TERMINAL 
CONTINUE TO PART 4, 



IF THERE IS A PRINTER AT YOUR TERMINAL 
ANSWER" THESE QUESTIONS 



DOES NOT APPLY 

STRONGLY DISAGREE 

DISAGREE 

NEITHER AGREE NOR DISAGREE 

■ AGREE _^ ~ 

B STRONGLY AGREE 

56. Ueing the printer te difficult 

57. The legibility of ■ copy from the printer ia edequete — 

58. The printer noise (a diffracting — 



OA 

p* ID 



PART 4: IMPROVING THE COMPUTER CATALOG 

INSTRUCTIONS: Select the response or responses that beet reflect your viewe about changaa that ahouid ba 
mada in tha computar catalog. 



59. WHAT ADDITIONAL FEATURES WOULD YOU 
LIKE THE COMPUTER CATALOG TO HAVE? 
(Select up to four) 

a. Providing slap by slap instructions 



h. 



Saarchtng by any word or words in a titla . . 
Searching by any word or words in a subject 

heading 

Limiting search results by date of 

publication 

Limiting seerch results by lenguege 

Ability to seerch by journel title 
ebbrevietions 

Ability to chenge the order in which items 

ere disployed j. 

Ability to view a list of words related to my 
search words • 

i. Ability to search for illust ret tons end 
bibliogrephiee 

j. Ability to seerch words that ere next to 
aech other 

k. Ability to print search results 

I. Ability to seerch a book's teble of contents, 
summery or index 

i. None 

i. Other--, 



.0 

.o 

.0 



60. WHICH COMPUTER CATALOG SERVICE 
IMPROVEMENTS WOULD YOU UKE THE 
LIBRARY TO MAKE? (Select up to four) 

a. More term t nets 

b. Terminele et locetions other than near 
the card catelog 

c. Terminals et pieces other then librery 
buildings 

d. A chert of commends posted et ti * 
terminal 

e. Menuel or brochure et the terminel — 

f. An instruction menuel for purchese — 

g. Troining sessions 

(Continued) 



60. (Continued) 

h. More stoff assistonce with the computer 
catelog 



•1 



i. Slide/tape/ceesotte treining progrem 

j. More of the librery's books in the computer 
catelog 

k. More kinds of moteriel such os journals, 
films, meps. etc. in the computer cetelog . . 

I. None 

m. Other:- 



12: 



WHAT ADDITIONAL KINDS OF MATERIALS 
WOULD YOU LIKE TO SEE INCLUDED IN THE 
COMPUTER CATALOG? 

(Select up to four) f 

a. More dissertetions 

More motion picture films 

More government publicettons 

More journel Or megezine titles 

More meps 

More monuscripts 

More music scores 

More newcpepers v 

More phonogreph records or tepes 

Mora technicel reports 

More of the librery's. older books 

None ." 



b. 
c. 
d. 
e. 

f. 

g 

h. 

i. 



k. 

I. 



m. Oth er:- 



JUST A FEW MORE 

QUESTIONS- 
PLEASE CONTINUE - 
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PART 5: COMPARING IfclE CARD CATALOG J}j£ COMPUTER QfiJAWQ 

INSTRUCTIONS: Compirt your experience of the card catalog with your experience of the com pu tat catalog. 
For aach atatamant indicata whathar tha CARD CATALOG IS SUPERIOR, tha COMPUTER CATALOG IS SU- 
PERIOR or there ia NO OIFFERENCE between tha two. "Card Catalog" includaa book cataloga and microfilm 
catalogs. 



62. In terms of ovarail searching spaad 

63. To saarch among all or most of tha books in tha library 
64 To aaaroh for a spacific book, journal or magaztna 

65. To laarch for books pubhshad in racant yaars 

66. To find a faw books on a topic ,i . .* 

67 To scan through savaral book titlas 

68 To iaarn to usa without assistanca 

69. To prapara a comprahansiva bibliography 



CARD 


THERE 


COMPUTER 


CATALOG 


IS NO 


CATALOG 


SUPERIOR 


DIFFERENCE 


SUPERIOR 


«_/ 
/— 

c 


o 
o 

O " 
0 


c 

0 

c k 















INSTRUCTIONS: Your rnpona 



PART 6: ABOUT YOURSELF 
i ara confidential. Plaaaa do not write your nama anywhara on this quastionnaira. 



■70. I USE THIS LIBRARY: 

■ a. Daily ..j 

1 b. Waakly - 

1 c. Monthly ; 

1 d. About four timas a year 

1 a. About onca a yaar - 

■ f. Not bafora today 

■71. I USE THIS COMPUTER CATALOG: 

■ a. Evary library visit - 

■ b. Almost avary visit * 

B c. Occasionally . * * 

■ d. Raraly * 

■ a. Not bafora today 

B 72. I US£ THIS LIBRARY'S BOOK. CARO OR 

* MICROFILM CATALOG: 

■ a. Evary visit ; 

■ b. Almost avary visit 

■ c. Occasionally 

■ d. Raraly , 

■ a. Not bafora today 

■73. THE USE OF COMPUTER TECHNOLOGY LIKE 

■ THE COMPUTER CATALOG IS: 

■ a. Vary familiar to ma — * > 

■ b. Somawhat familiar to ma . . » 

■ c. Somawhat unfamiliar to me 

■ d. Vary unfamiliar to ma 



74 



I USE A COMPUTER TERMINAL OTHER THAW 
THE LIBRARY'S TERMINALS: 

a. Daily • 

b. Waakly \ \ 

c. Monthly 

d. About four times a yaar 

a. About onca a yaar 

f. Navar ' 



75. 



FIRST BECAME AWARE OF THE COMPUTER 
CATALOG FROM: (Mark ail that apply) 

a. Saaing a terminal tn the library 

b. Library tour or orientation 

c. An article or written announcement 

d. A course instructor 

e. A friend or family member » 

f. Library staff ■. . * 

g. Other:-,. ; 



76. I LEARNEO HOW TO USE THE COMPUTER 
CATALOG: (Mark all that apply) 

a. By mysalf 

From a friend 

Using printed instructions 

Using instructions on the terminal screen . . . , 
From the library staff 

f. From a library course or orientation 

g. From a slida/tapa/cassette program 

h. Other-* ^ 



b. 
c. 
d. 
e. 
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77. 



MY AGE GROUP IS: _ 

a. 14 and under 

b. 15-19 years C 

c. 20-24 years 0 

d. 25-34 years O 

e. 35-44 years U 

f. 45-54 years C 

g. 55-64 years w 

h. 65 and over \J 



CONTINUE TO 78 



78. MARK YOUR CURRENT GRADE OR HIGHEST 
GRADE COMPLETED: 

Grade School or Elementery School . _ 

a. Up to five -> 

b. Six Z 

c. Seven . _ 

d. Eight > Z 

High School or Secondary School 

a. Nine J 

f. Ten Z 

g. Eleven ~ 

h. Twelve C 

College or University 

i. Thirteen - 

j. Fourteen _ 

k. Fifteen Z 

I. Sixteen Z 

m. Over sixteen _ 



IF YOU ARE COMPLETING THIS QUESTIONNAIRE 
AT A COLLEGE OR UNIVERSITY ANSWER THE 



NEXT a QUESTIONS^ - 



IF YOU ARE COMPLETING THIS QUESTIONNAIRE. 
AT A PUBUC LIBRARY STOP HERE AND RETURN 
THE QUESTIONNAIRE^ THANK. YOU FOR PAR- 
TICIPATING IN THIS STUDY OF THE COMPUTER" 

CATALOG. * 



79. THE CATEGORY THAT BEST DESCRIBES MY 
ACADEMIC AREA IS: 

Arts and Humanities O 

Natural Soencea O 

Social Sciences O 

Agriculture O 

Busineaa/Management O 

Education .0 

Engineering O 

Medicai/Heelth Sciences : . . C 

i. Law 1 O 

j. Major not declared O 

fc. Interdisciplinary O 

80. THE MAIN FOCUS OF MY ACADEMIC WORK 
AT THE PRESENT TIME IS: 

a. Course Work O 

b. Teeching O 

c. Research O 

d. Both Teaching end Reaearch '. ! -J 

e. Teaching and Course Work O 

f. Reseerch and Course Work % o 



81. 



MY PRESENT AFFILIATION WITH THIS COLLEGE 
OR UNIVERSITY IS: 



.0 



a. Undergraduate 

b. Graduate • maater's level 

c. Graduate * doctoral level w' 

d. Faculty , w» 

a. Nonfaculty Reaearch Staff 

f. Nonfaculty Teeching Staff «- 

g. College or University Staff ^' 

h. Other status v 

i. Alumnus or Alumna ^ 

j. No affiliation c O 



THIS COMPLETES THE QUESTIONNAIRE. 
PLEASE RETURN IT. 



THANK YOU FOR PARTICIPATING IN 
STUDY OF THE COMPUTER CATALOG. 



THIS 
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SUPPLEMENTARY QUESTIONNAIRE ITEMS 

89OC000 962CCCC 103 1 Z Z " . no 

90-0000 97 2 2 2 22 w Z Z Z V " m 

91 22 CCO 98 2 2 2 -2 Z 105 ~ Z-Z Z " , 112 

92 Z 2 2 0 0 99 2 -2 2 Z Z 106 Z I Z Z Z 113 

93 Z 2 2 2 2 100CCC 2 2 107 2 I 7 7 7 4 

94 2 2 2 22 101 2 2 2 2 Z 108 1 Z Z I Z 11s 

95 Z 2 2 2 2 102 Z Z 2 2 2 109 ' I ; : 7 76 
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APPENDIX D. PRETEST NONUSER QUESTIONNAIRE 



COUNCIL ON LIBRARY RESOURCES 

COMPUTER CATALOG STUDY 



OFFICE USE ONLY 
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QUESTIONNAIRE Z 



THE LIBRARY HAS^A COMPUTER CATALOG 
AND NOT EVERYONE HAS HAD A CHANCE TO 
USE IT YET, IF YOU HAVE NEVER USED IM£ 
COMPUTER CATALOG YOU CAN CONTRIBUTE 
TO THE QUALITY OF LIBRARY SERVICES BY 
COMPLETING THIS QUESTIONNAIRE. IT 
TAKES ABOUT FIVE MINUTES. YOUR RE- 
SPONSES ARE CONFIDENTIAL. PLEASE 00 
NOT WRITE YOUR NAME ANYWHERE ON 
THE QUESTIONNAIRE. THANK YOU. 



MARKING INSTRUCTIONS 



• USE A NO. 2 PENCIL ONLY 



• FILL THE CIRCLES COMPLETELY f£i 

• BE SURE TO ERASE 
CLEANLY ANY MARKS 
YOU WISH TO CHANGE 



• MAKE NO STRAY 
MARKS ON THIS 
QUESTIONNAIRE 



• • • I 
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part i: what xsm mm asqui iti£ 

COMPUTER CATALOG. 



INSTRUCTIONS : Please marfc the response that 
best describes how you view i com put* r catalog. 



"1. I HAVE NOT USEO THE COMPUTER CATALOG 

* UP TO NOW BECAUSE: 

i (Mark all that apply) 

> a. I do not lika to usa computers ^> 

i b. I did not know thera was a computer catalog ...C 
■ c. I do not know where it is 



o 



I have not had time to laarn how to use it . . 

t do not know if there are training sessions 
on how to use it 

There has not been any staff at the terminals 

to assist ma in using it 

The terminals ware ill in usa whan I wanted 

to use it 

I have not needed to use any library catalog ^ 

recently ^ 

i. The card catalog is easier to use w 

j; The card catalog contains more of the 

information I need ^ 

k. Otherx C 



.c 

.0 



MARK ONE ONLY IN QUESTIONS 2*5 

HOW MUCH TIME OQ YOU THINK IT TAKES TO 
LEARN TO USE THE COMPUTER CATALOG? _ 

a. 15 minutes or less ^ 

b. Between 15 minutes and 30 minutes w 

c Between 30 minutes and an hour <w. 

d. Between an hour and W of a. day w 

e. Between s h of a day and a day ~< 

f. A day or more 



HOW OIFFICULT OR EASY 00 YOU THINK IT 
WOULD BE TO LEARN TO USE THE COMPUTER 
CATALOG? 

a. Very difficult . « 

b. Somewhat difficult J 

c. Somewhat easy 

d. Very easy 



MY OVERALL OR GENERAL ATTITUOE TOWARO 
THE COMPUTER CATALOG IS: 

a. Very favorable 

b. Somewhat favorable 

c. Somewhat unfavorable 

d. Very unfavorable 

. HOW LIKELY ARE YOU TO USE THE COMPUTER 
CATALOG IN THE FUTURE? 
i. Very'hkely 

b. Somewhat" likely 

c. Somewhat unlikely 

d. Very unlikely 



PART 2: COMPARING THE CARD CATALOG 
AND THE COMPUTER CATALOG 

INSTRUCTIONS : Compere your experience of the 
card catalog with whet you expect or think a com- 
puter catalog does. For each statement indicate 
whether the CARO CATALOG IS PROBABLY 
SUPERIOR, the COMPUTER CATALOG IS PRO- 
BA8LY SUPERIOR or thera la PROBABLY NO 
OIFFERENCE betwaan the two. "Cerd Catalog" 
includes book catalogs and microfilm catalogs. 



COMPUTER CATALOG PROBABLY SUPERIOR 



PROBABLY IS NO OIFFERENCE 



CARO CATALOG PROBABLY SUPERIOR 



<03 



6. In terms of overall searching speed 

7. To search among all or most of the books in 
the library 

B. To search for a specific book, journal 

or magazine 

9. To search for books published in recent 
years 

10. To find a few-books on a topic 

1 1 . To scan through several book titles ... 

12. To learn to use without assistance 



13. To prepare a comprehensive bibliography . .p*j* D cm 
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PART 3: ABOUT YOURSELF 

INSTRUCTIONS : Your reeponsee art confidential. 
Plaase do not write your name anywhere on thia 
questionnaire. 



14. I USE THIS LIBRARY: 

i. Datiy ~ 

b. Weekly ' ^ 

c. Monthly x ' 

d. About four times a year 

e. About once a year O 

f. Not before today . . . O 

15. I USE THIS LIBRARY'S BOOK, CARD OR 
MICROFILM CATALOG: 

a. Every visit ^ 

b. Almost every visit O 

c. Occasionally ^ 

.d. Rarely ^ 

e. Not before today O 

16. THE USE OF COMPUTER TECHNOLOGY LIKE 
THE COMPUTER CATALOG IS: ^ 

a. Very familiar to me • • ^ 

b. Somewhat familiar to me ^ 

c. Somewhat unfamiliar to me ^ 

d. Very unfamiliar to me ^ 

17. I USE A COMPUTER TERMINAL OTHER THAN 
THE LIBRARY'S TERMINALS: 

a. Daily ^ 

b. Weekly ^ 

c. Monthly ^ 

d. About four times a year ^ 

e. About once a year • ^ 

f. Never .., ^ 

18. MY AGE GROUP IS: ^ 

a. 14 and under 

b. 15-1 9 years ^ 

c. 20-24 years ^ 

d. 25-34 years > ^ 

e. 35-44 years ^ 

f. 45-54 years O 

g. 55-64 years '."X 

h. 65 and over 



9. MARK YOUR CURRENT GRADE OR HIGHEST 
GRADE COMPLETED: 

Grade School or Elementary School 

a. Up to five 

b. Six 

c. Seven 

d. Eight - • ••• 

High School or Secondary School 

e. Nine • 

f. Ten ' • •• 

g. Eleven 

h. Twelve 

College or University 

i. Thirteen 

]. Fourteen 

k. Fifteen 

I. Sixteen • 

m. Over sixteen 



IF YOU ARE COMPLETING THIS QUESTIONNAIRE 
AT A COLLEGE OB UNIVERSfTY ANSWEH THE 
NEXT QUESTIONS. ^ - 



IF YOU ARE COMPLETING THIS QUESTION- 
NAIRE AT A eyjttU£ LIBRARY STOP HERE 
AND RETURN THE QUESTIONNAIRE. 
THANK YOU FOR PARTICIPATING IN THIS 
STUDY OF THE COMPUTER CATALOG. 



20. THE CATEGORY THAT BEST DESCRIBES MY 
ACADEMIC AREA IS: 

a. Arts and Humanities 

b. Natural Sciences 

c. Social Sciences 

d. Agriculture 

e. Business/Management 

f. Education ( 

g. Engineering 

h. Medical/Health Sciences 

t. Law 

j. Major not declared 

k. Interdisciplinary 



PLEASE CONTINUE ON THE BACK. PAGET- 
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1 21. THE MAIN FOCUS OF MY ACAOEMIC WORK 
» - AT THE PRESENT TIME IS: ^ 

■ a. Course Work »~ 

1 b. Teaching C 

• c. Research — 

1 d. Both Teaching and Research w 

■ «. Teaching and Course Work ^ 

1 f. Research and Course Work ^ 

»22. MY PRESENT AFFILIATION WITH THIS COLLEGE 

1 OR UNIVERSITY IS; 

1 a. Undergraduate ^ 

1 b. Graduate • master's level ~ 

■ c. Graduate • doctoral level — ^ 

■ d. Faculty ^ 

■ e. Nonfaculty Research Staff 'w 

■ f. Nonfaculty Teaching Staff ^ 

1 g. College or University Staff V 

■ h. Other status 

■ i, Alumnus or Alumna ^ 

■ j. Wo affiliation O 



THIS COMPLETES THE QUESTIONNAIRE. 
PLEASE RETURN IT. THANK YQU FOR 
PARTICIPATING IN THIS STUDY OF THE 
COMPUTER CATALOG. 
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